build a rest API service to provide market data for yourself
The framework I used is python flask, which is a micro service framework and very efficient in building rest API service. I considered doing this with aws API which can provide elastic scalability, however, your application will be tightly relying on aws in that way. I would keep my application independent and portable, so I only used the EC2 (Elastic Computer) service.
Prior to running the script, we need to install some modules:
flask, flask_limiter, flask_monitoringdashboard, gevent.
These modules are imported for rest API service:
from flask import Flask, request, g
from flask import render_template,jsonify
To provide data download function, we need to import this module in flask:
from flask import send_file
I don't want my server being flooded, so I import these modules to control the request limit:
from flask_limiter import Limiter
from flask_limiter.util import get_ipaddr
I use get_ipaddr
here to obtain the real IP address of the request to avoid overwhelming request, a request will pass through many agents in the route, this function can withdraw the initial IP address of the request.
flask does not support asynchronous request, so I import gevent for this purpose:
from gevent import monkey
from gevent.pywsgi import WSGIServer
monkey.patch_all()
To have a record of the API performance, we need to import this:
import flask_monitoringdashboard as dashboard
In another fold 'stock', I process all the things with basic request and location of files and web links on market data, we need to import these modules as well:
import aushare.stock.fundamental as td
from aushare.stock import cons as ct
I used a lot of pandas
to read data from .csv I have obtained and converted them to to the json format I need for private trading system.
https://github.com/chenlocus/aushare/haoserver.py is the python script you need to run.
Actually, flask is not a professional web server, so we can use gunicorn to run this server.
- install gunicorn:
pip install gunicorn
- run the server with gunicorn, '&' is to keep it a demo running in background, 'nohup' keeps the application running even if you exit the terminal.
nohup gunicorn -b 127.0.0.1: 12345 haoserver:app &
For the above command, it is mostly used in intranet or semi-product environment. In product environment, clients need to visit from extranet on port 8000 by default, and we need to process concurrent request, load balance. There is a powerful kit we can use which is Nginx, written by a Russian engineer.
Go to this file in EC2 instance using putty:
/etc/nginx/site-available/default
server {
listen 80 default_server;
listen [::]:80 default_server;
# SSL configuration
#
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
server_name _;
location / {
proxy_pass http://127.0.0.1:12345;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#try_files $uri $uri/ =404;
}
Then reload the setting to nginx, all done.
nginx -s reload