What is a NoSQL database? Quite simply, it is a database that isn’t SQL. SQL is a relational database made up of tables that contain rows of individual data items. Each row is split into columns. To take our User Data object, what we would have in a SQL database is a series of columns called: UserName, Password, RailroadLogoURL, RailroadName, CurrentTimetablePosition, MaximumTimetablePosition. Each column would have a type attached to it so: String, String, String, String, Integer, Integer.
To the “dyed in the wool” programmer, this all seems reasonable. However, if one of our users has a railroad name that exceeds 40 characters in length then he will have to truncate it to enter it into the system.
There would be tables for each of our data sets. The programmer would be expected to define primary keys and there would be relationships set up because the programmer would apply normalisation to the tables. This removes repeated data by referencing across tables.
So, why am I not doing this with the proposed system. Simply because the data isn’t complicated and doesn’t contain the type of repetition that makes normalisation a useful tool. The truth of this will become apparent as I move us through the process. There are a few basic types of NoSQL databases – key/value, document/ column and graph based. Rather than discuss all of these, I will just describe why I chose a key/value database as the way to go. See 3Pillar Global 2014.
Our data is basically simple. There is a user who has lists of locos, cars, towns, industries and so on. The system, when running in full mode will have lots of users. Each user will only have access to one set of data and no user’s data refers to other users data. Thus, storing the data for a user under a key gives fast and simple retrieval. I have built a key/value database previously – by using the Smalltalk built in functions to dump a binary record of an object and to retrieve it into its object form – through Microsoft’s file system. This proved to be very fast and very efficient. Rather than do that again, I investigated the leading key/value NoSQL database – Riak KV – and found that it had a REST (remember REST?) interface for which I could quickly build a Smalltalk interface. I only needed GET and PUT which were soon done and proved working.
The structure is very simple Basically, you only have to build a few little methods to access and use the database. Firstly, it will give you a new bucket if you just send data to one. Once the bucket is there, all you need are an “add to bucket” and “get from bucket” using keys – again, it will make a new key at the first time of putting. In Smalltalk, these look like the following:
add: anItem toBucket: aBucket usingKey: aKey | parameterString | parameterString := aBucket, '/', aKey. ^ self riakSstPost: parameterString content: anItem get: item fromBucket: aBucket ^ self riakGet:( self makeUrlWithItem: item path: aBucket) "Comment - ^ means answer the result"
The actual raw get and post are very simple
riakGet: dataRequest ^ (SstHttpClient fetch: dataRequest) contents reference riakSstPost: aString content: json | url tempClient | url := SstIpUrl fromString: RiakHost host hostName, RiakHost host path, aString. tempClient := (SstHttpClient forTransportScheme: 'httpl') startUp. ^ tempClient post: json typed: 'application/json' at: url
Notice that all the data is encoded into Json. Once in this format, we can send the data through the system without regard for content as that is recoverable at any stage – oh, and Riak likes Json.
So there you are – NoSQL101 is over for the day.