Store Arduino data to Firebase database [Howto]
[UPDATE] This is a very old project and at the time was the only solution I could think of. Now things changed and more powerful boards are available which solve the issue of https. Please have a look at this library instead https://github.com/FirebaseExtended/firebase-arduino.
The last few weeks I was playing with Firebase mostly because I wanted to get familiar with this technology. So at some point I thought that it will be a great idea to connect Arduino with Firebase. Imagine that you have a simple temperature sensor that measures the boiler’s water temperature every few minutes and you want to keep a history log of it in order to make some adjustments to thermostat. If you also want those measurements to be available online so you will be able to access them at any time then each sensor reading has to be stored somewhere.
One great way that I was also using for that kind of things is Xively. Originally it was Pachube , then Cosm and now Xively. I hope the guys don’t decide to change the name again! With Xively client for arduino you simply connect to their Api and you send them the readings. Its fast, its simple and they also have a free plan. But there are a few things that might not work for you. One of them is the limit of readings that you can store, at the moment in free plan you can have up to 3 months history. Another thing is that you can’t send data every 1 sec, so its impossible to store almost real time readings. And last but not least you need to consider that you are storing YOUR data in a database that you do not have control.
So what are the alternatives? Having your own sql database server is an overkill, not only because of the cost but also for maintenance reasons. For such simple projects you don’t have to spend a fortune. So here is where Firebase comes. The Firebase guys lets you connect to your own firebase database and do all the CRUD operations through simple calls to their api. You can visit their website and they will give you a 100mb database free. Which is enough to store at least a years readings. This is what I thought but quickly I came up with a huge problem. Firebase uses only HTTPS for security reasons and my Arduino Uno’s microprocessor don’t have enough power for that. In this link you can read what a member of firebase team says about the ssl support. The idea to bypass this issue is to proxy the requests through a server that can speak ssl. In order to achieve this I used the Firebase PHP library that you can find here. Think of it like this Arduino->Php website->Firebase. Once the data are in Firebase then you reatrive them in many different ways. Let’s start building our project and you will see how all the parts connect together.
First the hardware part. In my case I used the popular DS1820 temperature sensor. By connecting it to my Arduino Uno I was getting the room temperature readings. I also had to use an Ethernet shield. I had a spare W5100 so I made a small stack: Arduino Uno-Ethernet Sheild W5100-Prototyping Sheild. The circuit diagram is this:
And here is mine with all the parts connected:
Next step is to create your free firebase database in http://www.firebase.com. Then log into the database and go at the last menu item “Secrets”. From there you will get your secret token. Write it down as we will need it later. So now we have the hardware and the database ready. If you dont already have a domain, or sub domain go get one from your favorite registar and host the Firebase php library file in one of the thousand hosting providers that offer free plans. Personally I have a windows azure account so I just went there and created a new website.
Then by ftp I uploaded the firebaseLib.php:
<?php /** * Firebase PHP Client Library * * @author Tamas Kalman <[email protected]> * @link https://www.firebase.com/docs/rest-api.html * */ /** * Firebase PHP Class * * @author Tamas Kalman <[email protected]> * @link https://www.firebase.com/docs/rest-api.html * */ class Firebase { private $_baseURI; private $_timeout; private $_token; /** * Constructor * * @param String $baseURI Base URI * * @return void */ function __construct($baseURI = '', $token = '') { if (!extension_loaded('curl')) { trigger_error('Extension CURL is not loaded.', E_USER_ERROR); } $this->setBaseURI($baseURI); $this->setTimeOut(10); $this->setToken($token); } /** * Sets Token * * @param String $token Token * * @return void */ public function setToken($token) { $this->_token = $token; } /** * Sets Base URI, ex: http://yourcompany.firebase.com/youruser * * @param String $baseURI Base URI * * @return void */ public function setBaseURI($baseURI) { $baseURI .= (substr($baseURI, -1) == '/' ? '' : '/'); $this->_baseURI = $baseURI; } /** * Returns with the normalized JSON absolute path * * @param String $path to data */ private function _getJsonPath($path) { $url = $this->_baseURI; $path = ltrim($path, '/'); $auth = ($this->_token == '') ? '' : '?auth=' . $this->_token; return $url . $path . '.json' . $auth; } /** * Sets REST call timeout in seconds * * @param Integer $seconds Seconds to timeout * * @return void */ public function setTimeOut($seconds) { $this->_timeout = $seconds; } /** * Writing data into Firebase with a PUT request * HTTP 200: Ok * * @param String $path Path * @param Mixed $data Data * * @return Array Response */ public function set($path, $data) { return $this->_writeData($path, $data, 'PUT'); } /** * Pushing data into Firebase with a POST request * HTTP 200: Ok * * @param String $path Path * @param Mixed $data Data * * @return Array Response */ public function push($path, $data) { return $this->_writeData($path, $data, 'POST'); } /** * Updating data into Firebase with a PATH request * HTTP 200: Ok * * @param String $path Path * @param Mixed $data Data * * @return Array Response */ public function update($path, $data) { return $this->_writeData($path, $data, 'PATCH'); } /** * Reading data from Firebase * HTTP 200: Ok * * @param String $path Path * * @return Array Response */ public function get($path) { try { $ch = $this->_getCurlHandler($path, 'GET'); $return = curl_exec($ch); curl_close($ch); } catch (Exception $e) { $return = null; } return $return; } /** * Deletes data from Firebase * HTTP 204: Ok * * @param type $path Path * * @return Array Response */ public function delete($path) { try { $ch = $this->_getCurlHandler($path, 'DELETE'); $return = curl_exec($ch); curl_close($ch); } catch (Exception $e) { $return = null; } return $return; } /** * Returns with Initialized CURL Handler * * @param String $mode Mode * * @return CURL Curl Handler */ private function _getCurlHandler($path, $mode) { $url = $this->_getJsonPath($path); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_TIMEOUT, $this->_timeout); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $this->_timeout); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $mode); return $ch; } private function _writeData($path, $data, $method = 'PUT') { $jsonData = json_encode($data); $header = array( 'Content-Type: application/json', 'Content-Length: ' . strlen($jsonData) ); try { $ch = $this->_getCurlHandler($path, $method); curl_setopt($ch, CURLOPT_HTTPHEADER, $header); curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonData); $return = curl_exec($ch); curl_close($ch); } catch (Exception $e) { $return = null; } return $return; } }
And now the most importand file that you also have to upload to your hosting directory. Its the file that will respond to Arduino requests. I named it firebaseTest.php
<?php require_once 'firebaseLib.php'; // --- This is your Firebase URL $url = 'https://xxxxxx.firebaseio.com/'; // --- Use your token from Firebase here $token = 'xxxxxxxxxxxxxxx'; // --- Here is your parameter from the http GET $arduino_data = $_GET['arduino_data']; // --- $arduino_data_post = $_POST['name']; // --- Set up your Firebase url structure here $firebasePath = '/'; /// --- Making calls $fb = new fireBase($url, $token); $response = $fb->push($firebasePath, $arduino_data); sleep(2);
So now if you do a manual request to http://www.yourdomain.com/firbaseTest.php?arduino_data=21.00 the value 21.00 should be stored in your firebase database. And you should see it straight away. Pretty cool ah? Now the only thing that is left is to make the Arduino send the actually readings. So instead of the dummy “21.00” that we put manually before we will have the real temp readings. The arduino code is this one:
#include <SPI.h>
#include <Ethernet.h>
#include <OneWire.h>
int DS18S20_Pin = 2;
OneWire ds(DS18S20_Pin); // on digital pin 2
// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
char serverName[] = "ardufirebase.azurewebsites.net"; // name address of your domain
// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192,168,0,177);
int serverPort = 80;
EthernetClient client;
int totalCount = 0;
char pageAdd[64];
char tempString[] = "00.00";
// set this to the number of milliseconds delay
// this is 5 seconds
#define delayMillis 5000UL
unsigned long thisMillis = 0;
unsigned long lastMillis = 0;
void setup() {
Serial.begin(9600);
// disable SD SPI
pinMode(4,OUTPUT);
digitalWrite(4,HIGH);
// Start ethernet
Serial.println(F("Starting ethernet..."));
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
// no point in carrying on, so do nothing forevermore:
// try to congifure using IP address instead of DHCP:
Ethernet.begin(mac, ip);
}
digitalWrite(10,HIGH);
Serial.println(Ethernet.localIP());
delay(2000);
Serial.println(F("Ready"));
}
void loop()
{
thisMillis = millis();
if(thisMillis - lastMillis > delayMillis)
{
lastMillis = thisMillis;
float temperature = getTemp();
Serial.println(ftoa(tempString,temperature,2));
sprintf(pageAdd,"/firebaseTest.php?arduino_data=%s",ftoa(tempString,temperature,2));
if(!getPage(serverName,serverPort,pageAdd)) Serial.print(F("Fail "));
else Serial.print(F("Pass "));
totalCount++;
Serial.println(totalCount,DEC);
}
}
byte getPage(char *ipBuf,int thisPort, char *page)
{
int inChar;
char outBuf[128];
Serial.print(F("connecting..."));
if(client.connect(ipBuf,thisPort))
{
Serial.println(F("connected"));
sprintf(outBuf,"GET %s HTTP/1.1",page);
client.println(outBuf);
sprintf(outBuf,"Host: %s",serverName);
client.println(outBuf);
client.println(F("Connection: close\r\n"));
}
else
{
Serial.println(F("failed"));
return 0;
}
// connectLoop controls the hardware fail timeout
int connectLoop = 0;
while(client.connected())
{
while(client.available())
{
inChar = client.read();
Serial.write(inChar);
// set connectLoop to zero if a packet arrives
connectLoop = 0;
}
connectLoop++;
// if more than 10000 milliseconds since the last packet
if(connectLoop > 10000)
{
// then close the connection from this end.
Serial.println();
Serial.println(F("Timeout"));
client.stop();
}
// this is a delay for the connectLoop timing
delay(1);
}
Serial.println();
Serial.println(F("disconnecting."));
// close client end
client.stop();
return 1;
}
float getTemp(){
//returns the temperature from one DS18S20 in DEG Celsius
byte data[12];
byte addr[8];
if ( !ds.search(addr)) {
//no more sensors on chain, reset search
ds.reset_search();
return -1000;
}
if ( OneWire::crc8( addr, 7) != addr[7]) {
Serial.println("CRC is not valid!");
return -1000;
}
if ( addr[0] != 0x10 && addr[0] != 0x28) {
Serial.print("Device is not recognized");
return -1000;
}
ds.reset();
ds.select(addr);
ds.write(0x44,1); // start conversion, with parasite power on at the end
byte present = ds.reset();
ds.select(addr);
ds.write(0xBE); // Read Scratchpad
for (int i = 0; i < 9; i++) { // we need 9 bytes
data[i] = ds.read();
}
ds.reset_search();
byte MSB = data[1];
byte LSB = data[0];
float tempRead = ((MSB << 8) | LSB); //using two's compliment
float TemperatureSum = tempRead / 16;
return TemperatureSum;
}
char *ftoa(char *a, double f, int precision)
{
long p[] = {0,10,100,1000,10000,100000,1000000,10000000,100000000};
char *ret = a;
long heiltal = (long)f;
itoa(heiltal, a, 10);
while (*a != '\0') a++;
*a++ = '.';
long desimal = abs((long)((f - heiltal) * p[precision]));
itoa(desimal, a, 10);
return ret;
}
The above code is not that complicated as it may look. I used parts of the web client example code from arduino.cc. So in a few words what we are doing is we are making a request and we pass the current temperature reading. As you can see i set it to do the request every 5 seconds. The most difficult part for me was to convert the float value in string. If I didn’t do this I was getting an error message.
If all went well then you are almost ready. Your arduino is probably uploading the readings and you probably see your database filling. Here is a video from mine:
Now that you have a database full of readings you can use them as you like. Me, because this was only a small test project I created a simple html file and with some javascript and a jquery chart plugin I am just showing the current temperature from my room and a graph of it.
The index.html:
<html> <head> <script type='text/javascript' src='https://cdn.firebase.com/js/client/1.0.6/firebase.js'></script> <script language="javascript" type="text/javascript" src="http://code.jquery.com/jquery-2.1.0.min.js"></script> <script type="text/javascript" src="http://people.iola.dk/olau/flot/jquery.flot.js"></script> <script src="controller.js"></script> <link rel="stylesheet" type="text/css" href="bootstrap.min.css" /> </head> <body> <div style="width:800px; margin:0 auto;"> <div><h1 id='tempMsg' style='display:none;width:550px; margin:0 auto;margin-top: 50px;'>Current temp: <span id="currTemp"></span>°C</h1></div> <div id="chart1" style="width:600px;height:300px;"></div> </div> </body> <footer></footer> </html>
And the javascript file (controller.js)
$(document).ready(function() {
var tempsArray = [];
var tArray = [];
var sessionArray = [];
var currTemp = '';
var dataRef = new Firebase('https://arduino-test.firebaseio.com');
dataRef.on('value', function(snapshot) {
var t = snapshot.val();
var count = 0;
for (var key in t) {
if (t.hasOwnProperty(key)) {
var dt = [];
dt[0] = count;
dt[1] = parseFloat(t[key]);
tempsArray = [];
tempsArray.push(dt);
tArray = [];
tArray.push(dt[1]);
count++;
}
}
sessionArray.push(tempsArray[0])
//console.log(tempsArray)
$.plot($("#chart1"), [ sessionArray ]);
currTemp = tempsArray[0][1].toString();
$('#tempMsg').show();
$("#currTemp").text(currTemp);
});
});
Probably for your convenience you might want to upload also those to your hosting directory.
I hope you enjoyed this small project as I did. You can see the temperature in my room live in this link: http://ardufirebase.azurewebsites.net/
Special thanks to my colleague Fabrizio for his valuable help!
Thanks for reading
61 thoughts on “Store Arduino data to Firebase database [Howto]”