NAV Navbar

1. Introduction

Information on how to access the data via REST API can be found in the APIs section. Here, we'll provide a bit more information on the dataset and a simple example using the popular Pandas library in Python.

Our primary data source is ERA5 reanalysis dataset from ECMWF for historical weather data, available from 1980 to about 5 days ago. More recent past and forecast data uses GFS analysis and forecast dataset, made available by NCEP. These are public dataset and can be downloaded directly by the users but as the original data is stored in spatial format, extracting time-series can be an extremely slow process.

OikoLab has processed hundreds of terabytes of data to give you access this data in seconds rather than hours or even days, whether you require 3 months or 30 years of historical data, so that you can focus on your analysis rather than searching for data.

What is reanalysis?

Climate reanalyses combine vast observations from satellites, aircraft, land and sea based weather sensors with atmospheric models to generate consistent time series of multiple climate variables. ERA5 is the latest climate reanalysis produced by ECMWF, providing hourly data on many atmospheric, land-surface and sea-state parameters, on regular latitude-longitude grids at 0.25o x 0.25o lat/lon resolution.

alt text

2. Quick-start

Getting the data

Remember to use your own api_key.

import requests

r = requests.get('https://api.oikolab.com/weather',
                 params={'param': ['temperature','wind_speed'],
                         'start': '2010-01-01',
                         'end': '2018-12-31',
                         'lat': 23.1,
                         'lon': 114.1,
                         'api-key': '<your_api_key_here>'}
                 )

OikoWeather uses API keys to allow access to the API. Sign-up to get your API key.

The above command returns JSON data structured like this:

{'attributes': {
                'lat': 34.0544,
                'lon': -118.2439,
                'utc_offset': -8.0,
                'gfs_reference_time': '2020-09-24 18:00:00Z',
                 },
  'data': '{"columns": ["temperature (degC)","wind_speed (m\/s)"],
                       "index":[1262304000,1262307600,... ],
                       "data":[[16.59,7.61],[16.44,7.79]...]
          }'
}

Getting time-series weather data is easy. Simply make API request using the following parameters. The response data is provided in JSON format along with meta data.

Please note that the data is time-stamped for local standard timezone based on the location provided.

Parameter Option Type Description Notes
param required str Valid parameter from list below More than one parameter can be specified
start required str start datetime string per ISO 8601
end required str end datetime string per ISO 8601
freq optional str H for hourly data, or D for daily data. Defaults to H
location optional str location such as city name or zipcode for geocoding lookup
lat optional str latitude, if location is not provided
lon optional str longitude, if location is not provided

Working with Pandas

With json and pandas libraries, one can easily conver the JSON format to dataframe.

import json
import pandas as pd

weather_data = json.loads(r.json()['data'])
df = pd.DataFrame(index=pd.to_datetime(weather_data['index'], 
                                       unit='s'),
                  data=weather_data['data'],
                  columns=weather_data['columns'])

The data can be easily coverted to Pandas DataFrame, when converted, will look like this:

temperature (degC) wind_speed (m/s)
2010-01-01 00:00:00 16.59 7.61
2010-01-01 01:00:00 16.44 7.79
2010-01-01 02:00:00 16.27 7.60
2010-01-01 03:00:00 16.00 7.71
2010-01-01 04:00:00 15.56 7.62
... ... ...
2018-12-31 19:00:00 11.03 6.11
2018-12-31 20:00:00 10.70 6.34
2018-12-31 21:00:00 10.56 6.33
2018-12-31 22:00:00 10.38 6.38
2018-12-31 23:00:00 10.26 6.50

78888 rows × 2 columns

3. Weather Parameters

Available weather parameters are shown below, grouped into major categories for ease of reference. Please note that some parameters are only available for ERA5 (historical) data.

Group 1: Temperature

Parameter Min Max Unit Description Notes
temperature -80 +80 oC Drybulb temperature of air at 2m above the surface of land, sea or inland waters.
dewpoint_temperature -80 +80 oC The temperature to which the air at 2 metres above the surface of the Earth would have to be cooled for saturation to occur. Reference
wetbulb_temperature -80 +80 oC The temperature read by a thermometer covered in water-soaked cloth over which air is passed. Reference Calculated value.
urban_temperature -80 +80 oC Simulated temperature of urban area taking into account non-porous thermal mass,temperature, wind and radiation parameters.
humidex -80 +80 oC Humidex Index as defined by Canadian Meteorologists
soil_temperature_level_1 -80 +80 oC Soil temperature at level 1 (0 - 7cm), in the middle of layer.
soil_temperature_level_2 -80 +80 oC Soil temperature at level 2 (7 - 28cm), in the middle of layer.
soil_temperature_level_3 -80 +80 oC Soil temperature at level 3 (28 - 100cm), in the middle of layer.
soil_temperature_level_4 -80 +80 oC Soil temperature at level 4 (100 - 289cm), in the middle of layer.
sea_surface_temperature -80 +80 oC The temperature of sea water near the surface. In ERA5, this parameter is a foundation SST, which means there are no diurnal variations. Before September 2007, SST from the HadISST2 dataset is used and from September 2007 onwards, the OSTIA dataset is used.
skin_temperature -80 +80 oC The theoretical temperature that is required to satisfy the surface energy balance; it represents the temperature of the uppermost surface layer, which has no heat capacity. Skin temperature is calculated differently over land and sea.

Group 2: Wind

Parameter Min Max Unit Description Notes
windspeed 0 +60 m/s Wind speed at 10m above the surface. Calculated value.
wind_direction 0 360 deg Wind direction at 10m above the surface. Calculated value.
100m_windspeed 0 +60 m/s Wind speed at 100m above the surface. Calculated value.
100m_wind_direction 0 360 deg Wind direction at 100m above the surface. Calculated value.

Group 3: Radiation

Radiation parameters are accumulated values based on the previous time step, typically one hour or 3 hours in the case of 5 to 10 day forecast. ERA5 reanalysis values are originally provided in J/m2 and have been converted to W/m2.

Parameter Min Max Unit Description Notes
surface_solar_radiation 0 1300 W/m2 The amount of solar radiation (shortwave) that reaches a horizontal plane at the surface of the Earth. This parameter comprises both direct and diffuse solar radiation.
surface_thermal_radiation 0 1300 W/m2 The amount of thermal (longwave) radiation emitted by the atmosphere and clouds that reaches a horizontal plane at the surface of the Earth.
surface_direct_solar_radiation 0 1300 W/m2 The amount of direct solar radiation (shortwave) reaching the surface of the Earth. It is the amount of radiation passing through a horizontal plane. Currently only available for ERA5 data.
direct_normal_solar_radiation 0 1300 W/m2 The amount of solar radiation received per unit area normal to the direction of the sun. Calculated from solar zenith angle and surface direct solar radiation. Currently only available for ERA5 data.
surface_diffuse_solar_radiation 0 1300 W/m2 The amount of scattered solar radiation received per horizontal unit area on the surface. Calculated from surface and surface direct solar radiation. Currently only available for ERA5 data.

Group 4: Pressure, Humidity & Precipitation

Precipitation parameters are accumulated values based on the previous time step, typically one hour or 3 hours in the case of 5 to 10 day forecast. Original values provided in kg/m2 of water equivalent have been converted to mm of water equivalent.

Parameter Min Max Unit Description Notes
relative_humidity 0 1 - Relative humidity calculated from dewpoint temperature, drybulb temperature and pressure. Calculated value.
surface_pressure 50000 130000 Pa The pressure (force per unit area) of the atmosphere at the surface of land, sea and inland water.
total_cloud_cover 0 1 - Single level field calculated from the cloud occurring at different model levels through the atmosphere.
total_precipitation 0 50 mm The accumulated liquid and frozen water, comprising rain and snow, that falls to the Earth's surface. It is the sum of large-scale precipitation and convective precipitation.
snowfall 0 50 mm The accumulated snow that falls to the Earth's surface. It is the sum of large-scale snowfall and convective snowfall. Currently only available for ERA5 data

4. Frequently Asked Questions

How is the data generated?

We use ERA5 Reanalysis data from ECMWF for historical data up to about 5~10 days prior to the current date. This data is combined with GFS data for recent history and up to 10 days forecast. ERA5 data is update daily, and GFS data is updated every 6 hours.

The model data resolution is 0.25o x 0.25o (lat x lon) and we perform spatial (bilinear) interpolation based on the location provided. If the location is provided as a text string, we use third-party geo-coding service to look up the latitude and longitude and return this value for your information. If you want to be more precise, please provide latitude and longitude values directly.

How does this data compare to weather station data?

Both ERA5 and GFS are data output from physics-based simulation models, which means that they ingest millions of observations including weather station data to produce outcome consistent with weather-physics. This also corrects of any missing gaps or biases in individual observations, so that overall consistency is better and parameters that are not typically included in weather station reporting, such as solar radiation and ground temperature can be derived.

How does ERA5 reanalysis data compare with MERRA-2?

ERA5 has about 4 times the spatial resolution when compared with MERRA-2 (0.25o x 0.25o vs. 0.5o x 0.625o (lat/lon)).

How does this compare with other weather data providers?

Unlike make other providers, we don't claim to provide more accurate results than what NOAA and ECMWF are able to do - we recognize the complexity and the chaotic nature of weather models. Our goal is to make the data more accessible by standing on the shoulders giants.

What are the limitations of these models?

As we use the same GFS and ERA5 data, we are bound by similar limitations of these models - for instance, the models may be less accurate in mountainous or coastal regions and not suitable analyzing extreme weather events such as storms.

Do you have ERA5-land data?

Coming soon! We will be adding the first batch of ERA5-land dataset (9km resolution) soon.

How do I create an AMY EPW file for EnergyPlus?

We suggest downloading the parameters and using scripting tools such as Ladybug to generate EPW files.

Don't see the parameters you need?

Please reach out to us at [email protected] - we'd be more than happy to help.