Data Flow

I am now at the point where I have data flowing through the system from front to back and front again. At this point it is only display data, which is OK because some items can only be displayed. They are not configurable in the system. Why, you may ask? Well, there is only so much that I can do in the time given. The periods between Tutor Marked Assignments (TMAs) may seem long but the size of the assignment grows with each milestone (or gravestone, depending on how you see it). The next assignment is due on 13th April which is weeks away. However, I have to have 4,000 words written, reviewed, corrected and ready for the tutor before then, plus I have to manage a TMA for TM352 (the Web/Mobile/Cloud course that I am also doing). Incidentally, my wife would like me to have some sort of life as well – go shopping, read a book, do some work on my railroad, make a model, etc. Some hopes!

Anyway back to the technical side. What I am about to describe is how the system processes a request for data. The process is quite complicated and it is made more complex because I have decided to construct it so that, in theory, it all could run on separate Virtual Machines (VMs) in the cloud. The first requirement is to conceptually separate the activities. These are:

  1. Manage the request and display of the data (Display Manager)
  2. Manage the conversion of requests into data requests and makes database queries (Web Control manager)
  3. Manage the database queries and respond with formatted data (Database manager)

I will deal with the three concepts describing the actual flow as I go.

Display Manager

The display manager knows nothing about anything so everything has to be given to it on request. It starts of with a url arriving in the format –

At the moment, I am only testing so the URL above doesn’t work! I am working on “localhost” which means a single machine, for now.  The next bit – 8080- is a port on the computer. All network requests, etc. work through ports (your e-mail goes through port 25 out and port 110 in). This means that there has to be a server listening on that port to accept the request. WSDisplayPage is a chunk of Smalltalk – actually an object class – that accepts the request. Everything following the ? is information to the class to enable it to decide what to do. “demo” is the test user name and “trains” is the display that we want to show – “trains” being a list of “Trains” held in the database.

Once WSDisplay gets this data, it must decide what to do with it. It strips out the user and the request (trains) and constructs another URL that goes to the Web Control Manager. This will be in the form:

Note that it is using port 8081 – this is because if we had all of this on separate VMs, the router managing the incoming traffic must know to send this request to a different VM.

So, it has gone and the response has come back. In fact the server at the controller side just created some HTML as if it was answering a normal web browser and sent it back to the requesting url. What comes back is a whole lot of Json (remember Json – we talked about that earlier) that describes all of the trains that we have stored. This is formatted up into a standard web display and provided to the requesting browser in the normal fashion. (See Appendix 1)

Web Control Manager (WCM)

The WCM listens on its port and responds to any request as though it was a web browser (is this sounding familiar?) WCGetData is only one of its classes as it has other types of requests that need to be serviced. WCGetData formats up the request into a database request.

Notice that the address has changed again. This time, we have a simple internal IP address (no domain name) as the database server will never be exposed to the outside web. Riak listens on its own port, which happens to be 8098. The above request results in the WCM receiving back a Json that is the complete data held for the user – demo. It strips out just the “trains” and passes this, still in Json format, back to the Display Manager – again as though it was a web browser request.

Database Manager

This bit is supplied as a total entity and can only be configured (or corrupted!) by changing configuration settings. Riak is a very complex instrument if you want it to be but it is also a very simple Key/Value (KV) database. Tell it a bucket, tell it what to put in that bucket under which key and it will do it. Ask for it back and it comes. Couldn’t be simpler. I have written a KV database myself and could have used that for this project. Riak has one very good extra feature. If you put three Riak servers on different machines, link them together and set them going, the data will be replicated automatically and if one server goes down, the database keeps going!

So, on the above url being received, Riak goes to the required bucket, retrieves the requested data – which, of course, is in Json format and returns it as an html response. As we now know, this goes all the way up the line being manipulated as it goes to enable the Web Browser to display the information.

So that is GET. I will deal with saving data with a POST in another entry.

Appendix 1

I say that the Display Manager ” knows nothing about anything so everything has to be given to it on request”. This is done by some clever Smalltalk. Smalltalk has a type called a Symbol. This has a special feature  in that it can be changed into a method name or even a class. Imagine that we had a class called TM470Loco which has instance variables called ‘road’ and ‘locoNumber’. The following needs a bit of thought but I will explain it.

| aLoco|
aLoco := #TM470Loco asClass new 
                     perform: 'road:' asSymbol with: 'SNE'.
^ aLoco road

TM470 is a defined class and sending the message asClass to the symbol #TM470Loco turns it into a class name. Sending new to the class creates a new instance of TM470Loco called aLoco. Perform: with: is a message to the new object to put ‘SNE’ into its instance variable called “road” – the : after road makes it into a setter, without the : it would be a getter and can only answer a value. The caret ^ tells the script to “answer” the contents of the road instance variable of the new object called aLoco.

It is this mechanism that enables me to say that the Display Manager knows nothing about the system. Every class knows how it is to be displayed and which methods are used in that display. The Display Manager uses the above mechanism to build a display without knowing anything about any of the data that it receives. Clever wot!

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s