External Weather Data from API, Sharing Python Code

Discussion

Home Discussion Board KDD CUP of Fresh Air External Weather Data from API, Sharing Python Code

GorakGarak

External Weather Data from API, Sharing Python Code

posted in KDD CUP of Fresh Air

2018-04-08 03:58

# Why using weather forecasting data?

After I've done some EDA, I found out that the future weather data would really help to predict the upcoming PM level.

openweathermap.org provides valuable weather data.  I have made simple python code to fetch those weather data.

# What you should do first
First, visit openweather.org and sign in. You will find your api key in your personal profile.

This website also provides various data such as air pollution, UV alerts, etc. If you are interested, check API docs.

# JSON weather file

API returns JSON raw data.

```
"list": [
        {
            "dt": 1523167200,
            "main": {
                "temp": 281.36,
                "temp_min": 281.36,
                "temp_max": 281.461,
                "pressure": 995.6,
                "sea_level": 1034.77,
                "grnd_level": 995.6,
                "humidity": 98,
                "temp_kf": -0.1
            },
            "weather": [
                {
                    "id": 500,
                    "main": "Rain",
                    "description": "light rain",
                    "icon": "10n"
                }
            ],
            "clouds": {
                "all": 20
            },
            "wind": {
                "speed": 1.5,
                "deg": 311.501
            },
            "rain": {
                "3h": 0.030000000000001
            },
            "sys": {
                "pod": "n"
            },
            "dt_txt": "2018-04-08 06:00:00"
        },
```

# Sample python code

you just put csv with city zipcode list, then temperature will be pop out.

Because I have used this code for fetching Non-English characters, there are some unnecessary code related to UTF-8. If you are dealing with Chinese chracter, these will help you.

you can change the code at your will.

```
# -*- coding: utf-8 -*-

import os
import sys
import urllib.request
import json
import csv
import time

# 2018.04.08 - Read city list from csv
def read_file():
    dic = dict()

## <REPLACE XXXX with your city list> ##
    f = open('XXXX.csv', 'r', encoding='utf-8', newline='') 
    rdr = csv.reader(f)
    for line in rdr:
        zipcode = line[0]
        dic[zipcode]=0 
    f.close()    
    return dic

def get_temp(cityzipcode):

encText = urllib.parse.quote(city.encode('utf-8'))
    countrycode = "cn"

##  <REPLACE this key with your own api key. openweathermap.org > ## 
    apiKey = "" 
    url = "http://api.openweathermap.org/data/2.5/forecast?zip=" + cityzipcode + "," + countrycode + "&appid=" + apiKey 
    
    request = urllib.request.Request(url)
    response = urllib.request.urlopen(request)
    rescode = response.getcode()

if(rescode==200):
        response_body = response.read()
        result = response_body.decode('utf-8')
        raw = json.loads(result)
        try :
            # get temperature from returned json list
            # <REPLACE node with this if needed>
            temp = raw['list'][0]['temp'] 
        except :
            temp = 0 
        return temp

else:
        print("Error Code:" + rescode)
        return 0

def write_file(dic, file_name):
    with open(file_name, 'w') as f:
        fieldnames = ['City', 'Temp']
        writer = csv.DictWriter(f, fieldnames=fieldnames)
        writer.writeheader()
        data = [dict(zip(fieldnames, [k, v])) for k, v in dic.items()]
        writer.writerows(data)

print("---------start---------------")

# read city list from file. you can just hardcode the city list in this python file.
dic = read_file()

try :
    # put score into the dictionary dic, if exists.
    for city in dic:
        temp = get_temp(city)
        print(f"{city} - {temp}")
        dic[city] = temp
        time.sleep(1) # API will possibly block your request without this. replace it if you want.

finally : # sometimes, 500 error occurs.
    # when all done, save it to csv
    write_file(dic, "temp.csv")

print("-----------end-------------")
```

Bye! Happy coding!

LET'S PREVENT THOSE LITTLE EVIL DUSTS AND LIVE LONGER!

23 comments

KDD18 Organizer

JeongYoon

2018-04-14 16:38

yazaemon

2018-04-27 17:06