Objectives

Extend the pacemaker application to include Activity and Location classes + associated commands. Once these are in place, incorporate a serialization mechanism to enable users & activities to be persisted to a file. We will then try to generalize this mechanism, which will enable us to experiment with alternative serialization formats.

Activity & Location classes

These are solution classes to last weeks exercises:

Activity

package models;

import static com.google.common.base.MoreObjects.toStringHelper;

import java.util.ArrayList;
import java.util.List;

import com.google.common.base.Objects;

public class Activity 
{ 
  static Long   counter = 0l;

  public Long   id;

  public String type;
  public String location;
  public double distance;

  public List<Location> route = new ArrayList<>();

  public Activity()
  {
  }

  public Activity(String type, String location, double distance)
  {
    this.id        = counter++;
    this.type      = type;
    this.location  = location;
    this.distance  = distance;
  }

  @Override
  public String toString()
  {
    return toStringHelper(this).addValue(id)
                               .addValue(type)
                               .addValue(location)
                               .addValue(distance)
                               .addValue(route)
                               .toString();
  }

  @Override  
  public int hashCode()  
  {  
     return Objects.hashCode(this.id, this.type, this.location, this.distance);  
  } 
}

Location

package models;

import static com.google.common.base.MoreObjects.toStringHelper;

import com.google.common.base.Objects;

public class Location
{
  static Long   counter = 0l;

  public Long  id;
  public float latitude;
  public float longitude;

  public Location()
  {
  }

  public Location (float latitude, float longitude)
  {
    this.id        = counter++;
    this.latitude  = latitude;
    this.longitude = longitude;
  }

  @Override
  public String toString()
  {
    return toStringHelper(this).addValue(id)
                               .addValue(latitude)
                               .addValue(longitude)                              
                               .toString();
  }

  @Override  
  public int hashCode()  
  {  
     return Objects.hashCode(this.id, this.latitude, this.longitude);  
  } 
}

API + Commands

These are the new features for PacemakerAPI

  public void createActivity(Long id, String type, String location, double distance)
  {
    Activity activity = new Activity (type, location, distance);
    Optional<User> user = Optional.fromNullable(userIndex.get(id));
    if (user.isPresent())
    {
      user.get().activities.put(activity.id, activity);
      activitiesIndex.put(activity.id, activity);
    }
  }

  public Activity getActivity (Long id)
  {
    return activitiesIndex.get(id);
  }

  public void addLocation (Long id, float latitude, float longitude)
  {
    Optional<Activity> activity = Optional.fromNullable(activitiesIndex.get(id));
    if (activity.isPresent())
    {
      activity.get().route.add(new Location(latitude, longitude));
    }
  }

...and these are the new command implementations:

  @Command(description="Add an activity")
  public void addActivity (@Param(name="user-id")  Long   id,       @Param(name="type") String type, 
                           @Param(name="location") String location, @Param(name="distance") double distance)
  {
    Optional<User> user = Optional.fromNullable(paceApi.getUser(id));
    if (user.isPresent())
    {
      paceApi.createActivity(id, type, location, distance);
    }
  }

  @Command(description="Add Location to an activity")
  public void addLocation (@Param(name="activity-id")  Long  id,   
                           @Param(name="latitude")     float latitude, @Param(name="longitude") float longitude)
  {
    Optional<Activity> activity = Optional.fromNullable(paceApi.getActivity(id));
    if (activity.isPresent())
    {
      paceApi.addLocation(activity.get().id, latitude, longitude);
    }
  }

The above code uses a feature of the Guava library that we have not explained yet in class. This is the `Optional' class:

• https://github.com/google/guava/wiki/UsingAndAvoidingNullExplained

This is a utility that helps our code avoid the possibility of Null Pointer Violations. We will discuss this in detail is a future class.

Try these out now. The command shell we are using has a facility for running a script - the `!rs' command. So, if you save this file somewhere as 'test.script':

cu homer simpsom homer@simpson.com secret
cu marge simpson marge@simpson.com secret
aa 0 walk fridge .001
aa 0 walk bar 1.0
aa 0 run work 2.2
aa 1 walk shop 2.5
aa 1 cycle shop 4.5
al 3 23.3 32.3
al 3 23.3 32.5
al 3 23.3 32.6

and then run it:

Welcome to pcemaker-console - ?help for instructions
pc> !rs test.script
pc$ pc$ pc$ pc$ pc$ pc$ pc$ pc$ pc$ pc$ pc> gu
[User{0, homer, simpsom, secret, homer@simpson.com, {0=Activity{0, walk, fridge, 0.001, []}, 1=Activity{1, walk, bar, 1.0, []}, 2=Activity{2, run, work, 2.2, []}}}, User{1, marge, simpson, secret, marge@simpson.com, {3=Activity{3, walk, shop, 2.5, [Location{0, 23.3, 32.3}, Location{1, 23.3, 32.5}, Location{2, 23.3, 32.6}]}, 4=Activity{4, cycle, shop, 4.5, []}}}]
pc>

Commit these changes with a suitable message

Serialization

We would like our system to remember our user and activity data between sessions. In lab 1 we explored the xstream component briefly. We will reuse it here to save the entire model on exit from the application, and reload on startup.

Here is a refactoring of the main method to bring in load/store to the application life cycle:

  public static void main(String[] args) throws Exception
  {
    Main main = new Main();

    File  datastore = new File("datastore.xml");
    if (datastore.isFile())
    {
      main.paceApi.load(datastore);
    }

    Shell shell = ShellFactory.createConsoleShell("pm", "Welcome to pacemaker-console - ?help for instructions", main);
    shell.commandLoop();

    main.paceApi.store(datastore);
  }

In the above main method, we check to see if the file 'datastore.xml' exists. If so, we attempt to load a model from it. If not, we proceed with a blank model.

On exit (i.e. when 'exit' is typed from the console) we ask the model to store itself.

Thus we need two new methods in PacemakerApi:

• load
• store

Here they are:

  @SuppressWarnings("unchecked")
  public void load(File file) throws Exception
  {
    ObjectInputStream is = null;
    try
    {
      XStream xstream = new XStream(new DomDriver());
      is = xstream.createObjectInputStream(new FileReader(file));
      userIndex       = (Map<Long, User>)     is.readObject();
      emailIndex      = (Map<String, User>)   is.readObject();
      activitiesIndex = (Map<Long, Activity>) is.readObject();
    }
    catch(EOFException e) 
    {
    }
    finally
    {
      if (is != null)
      {
        is.close();
      }
    }
  }

  public void store(File file) throws Exception
  {
    XStream xstream = new XStream(new DomDriver());
    ObjectOutputStream out = xstream.createObjectOutputStream(new FileWriter(file));
    out.writeObject(userIndex);
    out.writeObject(emailIndex);
    out.writeObject(activitiesIndex);
    out.close(); 
  }

Take some time to read these now - and try them out.

In particular, see if you can locate the 'datastore.xml' file after the app has exited. You will need to 'refresh' your workspace for it to appear in eclipse project explorer.

If all goes to plan, and say you run the script we explored earlier, then in eclipse you should be able to examine the datastore file in an xml editor. It could look like this:

Commit these changes with a suitable message

Generalizing the Serializer

We have implemnted an xstream based serialization of our model. However, we may be interested in revising this later, selecting a different format or even a different serialisation component. Lets try to abstract the serializer feature a little so we can prepare for such a switch.

First, introduce into the utils package a new interface:

package utils;

public interface Serializer
{
  void push(Object o);
  Object pop();
  void write() throws Exception;
  void read() throws Exception;
}

... and then an implementation of this interface, using the xtream library we have been using:

package utils;

import com.thoughtworks.xstream.XStream;
import com.thoughtworks.xstream.io.xml.DomDriver;

import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.Stack;

public class XMLSerializer implements Serializer
{

  private Stack stack = new Stack();
  private File file;

  public XMLSerializer(File file)
  {
    this.file = file;
  }

  public void push(Object o)
  {
    stack.push(o);
  }

  public Object pop()
  {
    return stack.pop(); 
  }

  @SuppressWarnings("unchecked")
  public void read() throws Exception
  {
    ObjectInputStream is = null;

    try
    {
      XStream xstream = new XStream(new DomDriver());
      is = xstream.createObjectInputStream(new FileReader(file));
      Object obj = is.readObject();
      while (obj != null)
      {
        stack.push(obj);
        obj = is.readObject();
      }
    }
    finally
    {
      if (is != null)
      {
        is.close();
      }
    }
  }

  public void write() throws Exception
  {
    ObjectOutputStream os = null;

    try
    {
      XStream xstream = new XStream(new DomDriver());
      os = xstream.createObjectOutputStream(new FileWriter(file));
      while (!stack.empty())
      {
        os.writeObject(stack.pop());  
      }
    }
    finally
    {
      if (os != null)
      {
        os.close();
      }
    }
  }
}

Now rework the Main class to attempt to load from the datastore in a constructor for Main:

  public Main() throws Exception
  {
    File  datastore = new File("datastore.xml");
    Serializer serializer = new XMLSerializer(datastore);

    paceApi = new PacemakerAPI(serializer);
    if (datastore.isFile())
    {
      paceApi.load();
    }
  }

Note that we only load if there is a file is available. The main method can then be simplified:

  public static void main(String[] args) throws Exception
  {
    Main main = new Main();

    Shell shell = ShellFactory.createConsoleShell("pm", "Welcome to pacemaker-console - ?help for instructions", main);
    shell.commandLoop();

    main.paceApi.store();
  }

Finally, our PacemakerAPI class can now make use of the serializer instead of getting involved in the work of serialization itself:

public class PacemakerAPI
{
  //...

  private Serializer serializer;

  public PacemakerAPI(Serializer serializer)
  {
    this.serializer = serializer;
  }

  @SuppressWarnings("unchecked")
  public void load() throws Exception
  {
    serializer.read();
    activitiesIndex = (Map<Long, Activity>) serializer.pop();
    emailIndex      = (Map<String, User>)   serializer.pop();
    userIndex       = (Map<Long, User>)     serializer.pop();
  }

  public void store() throws Exception
  {
    serializer.push(userIndex);
    serializer.push(emailIndex);
    serializer.push(activitiesIndex);
    serializer.write(); 
  }
  //...
}

You will notice that you have a lot of import warnings; remove any unused imports.

If you have an existing datastore.xml file in your project, delete this.

You should be in a position now to test. Remember, if you have a script file like this:

cu homer simpsom homer@simpson.com secret
cu marge simpson marge@simpson.com secret
aa 0 walk fridge .001
aa 0 walk bar 1.0
aa 0 run work 2.2
aa 1 walk shop 2.5
aa 1 cycle shop 4.5
al 3 23.3 32.3
al 3 23.3 32.5
al 3 23.3 32.6

Then you can just run this script using the !rs command, and the 'gu' command should print the model:

Welcome to pacemaker-console - ?help for instructions
pm> !rs test.script
pm$ pm$ pm$ pm$ pm$ pm$ pm$ pm$ pm$ pm$ pm> gu
[User{0, homer, simpsom, secret, homer@simpson.com, {0=Activity{0, walk, fridge, 0.001, []}, 1=Activity{1, walk, bar, 1.0, []}, 2=Activity{2, run, work, 2.2, []}}}, User{1, marge, simpson, secret, marge@simpson.com, {3=Activity{3, walk, shop, 2.5, [Location{0, 23.3, 32.3}, Location{1, 23.3, 32.5}, Location{2, 23.3, 32.6}]}, 4=Activity{4, cycle, shop, 4.5, []}}}]
pm>

The above session will generate the following xml file:

<object-stream>
  <map>
    <entry>
      <long>0</long>
      <models.Activity>
        <id>0</id>
        <type>walk</type>
        <location>fridge</location>
        <distance>0.001</distance>
        <route/>
      </models.Activity>
    </entry>
    <entry>
      <long>1</long>
      <models.Activity>
        <id>1</id>
        <type>walk</type>
        <location>bar</location>
        <distance>1.0</distance>
        <route/>
      </models.Activity>
    </entry>
    <entry>
      <long>2</long>
      <models.Activity>
        <id>2</id>
        <type>run</type>
        <location>work</location>
        <distance>2.2</distance>
        <route/>
      </models.Activity>
    </entry>
    <entry>
      <long>3</long>
      <models.Activity>
        <id>3</id>
        <type>walk</type>
        <location>shop</location>
        <distance>2.5</distance>
        <route>
          <models.Location>
            <id>0</id>
            <latitude>23.3</latitude>
            <longitude>32.3</longitude>
          </models.Location>
          <models.Location>
            <id>1</id>
            <latitude>23.3</latitude>
            <longitude>32.5</longitude>
          </models.Location>
          <models.Location>
            <id>2</id>
            <latitude>23.3</latitude>
            <longitude>32.6</longitude>
          </models.Location>
        </route>
      </models.Activity>
    </entry>
    <entry>
      <long>4</long>
      <models.Activity>
        <id>4</id>
        <type>cycle</type>
        <location>shop</location>
        <distance>4.5</distance>
        <route/>
      </models.Activity>
    </entry>
  </map>
  <map>
    <entry>
      <string>homer@simpson.com</string>
      <models.User>
        <id>0</id>
        <firstName>homer</firstName>
        <lastName>simpsom</lastName>
        <email>homer@simpson.com</email>
        <password>secret</password>
        <activities>
          <entry>
            <long>0</long>
            <models.Activity>
              <id>0</id>
              <type>walk</type>
              <location>fridge</location>
              <distance>0.001</distance>
              <route/>
            </models.Activity>
          </entry>
          <entry>
            <long>1</long>
            <models.Activity>
              <id>1</id>
              <type>walk</type>
              <location>bar</location>
              <distance>1.0</distance>
              <route/>
            </models.Activity>
          </entry>
          <entry>
            <long>2</long>
            <models.Activity>
              <id>2</id>
              <type>run</type>
              <location>work</location>
              <distance>2.2</distance>
              <route/>
            </models.Activity>
          </entry>
        </activities>
      </models.User>
    </entry>
    <entry>
      <string>marge@simpson.com</string>
      <models.User>
        <id>1</id>
        <firstName>marge</firstName>
        <lastName>simpson</lastName>
        <email>marge@simpson.com</email>
        <password>secret</password>
        <activities>
          <entry>
            <long>3</long>
            <models.Activity>
              <id>3</id>
              <type>walk</type>
              <location>shop</location>
              <distance>2.5</distance>
              <route>
                <models.Location>
                  <id>0</id>
                  <latitude>23.3</latitude>
                  <longitude>32.3</longitude>
                </models.Location>
                <models.Location>
                  <id>1</id>
                  <latitude>23.3</latitude>
                  <longitude>32.5</longitude>
                </models.Location>
                <models.Location>
                  <id>2</id>
                  <latitude>23.3</latitude>
                  <longitude>32.6</longitude>
                </models.Location>
              </route>
            </models.Activity>
          </entry>
          <entry>
            <long>4</long>
            <models.Activity>
              <id>4</id>
              <type>cycle</type>
              <location>shop</location>
              <distance>4.5</distance>
              <route/>
            </models.Activity>
          </entry>
        </activities>
      </models.User>
    </entry>
  </map>
  <map>
    <entry>
      <long>0</long>
      <models.User>
        <id>0</id>
        <firstName>homer</firstName>
        <lastName>simpsom</lastName>
        <email>homer@simpson.com</email>
        <password>secret</password>
        <activities>
          <entry>
            <long>0</long>
            <models.Activity>
              <id>0</id>
              <type>walk</type>
              <location>fridge</location>
              <distance>0.001</distance>
              <route/>
            </models.Activity>
          </entry>
          <entry>
            <long>1</long>
            <models.Activity>
              <id>1</id>
              <type>walk</type>
              <location>bar</location>
              <distance>1.0</distance>
              <route/>
            </models.Activity>
          </entry>
          <entry>
            <long>2</long>
            <models.Activity>
              <id>2</id>
              <type>run</type>
              <location>work</location>
              <distance>2.2</distance>
              <route/>
            </models.Activity>
          </entry>
        </activities>
      </models.User>
    </entry>
    <entry>
      <long>1</long>
      <models.User>
        <id>1</id>
        <firstName>marge</firstName>
        <lastName>simpson</lastName>
        <email>marge@simpson.com</email>
        <password>secret</password>
        <activities>
          <entry>
            <long>3</long>
            <models.Activity>
              <id>3</id>
              <type>walk</type>
              <location>shop</location>
              <distance>2.5</distance>
              <route>
                <models.Location>
                  <id>0</id>
                  <latitude>23.3</latitude>
                  <longitude>32.3</longitude>
                </models.Location>
                <models.Location>
                  <id>1</id>
                  <latitude>23.3</latitude>
                  <longitude>32.5</longitude>
                </models.Location>
                <models.Location>
                  <id>2</id>
                  <latitude>23.3</latitude>
                  <longitude>32.6</longitude>
                </models.Location>
              </route>
            </models.Activity>
          </entry>
          <entry>
            <long>4</long>
            <models.Activity>
              <id>4</id>
              <type>cycle</type>
              <location>shop</location>
              <distance>4.5</distance>
              <route/>
            </models.Activity>
          </entry>
        </activities>
      </models.User>
    </entry>
  </map>
</object-stream>

However - there is something profoundly problematic with the above. What is it that is wrong (it is difficult to spot)?

Object References and Serialization

The problem with our current serializer is that the three Maps serialized are completely independent - even though the maps in memory prior to serialization contain shared objects. This is apparent from this structural view here:

Correcting this is relatively straightforward. Object referential context is only preserved over a single writeObject operation - we have been doing several on our serializer - see the relevant fragments of the read and write methods:

write

      while (!stack.empty())
      {
        os.writeObject(stack.pop());  
      }

read

      Object obj = is.readObject();
      while (obj != null)
      {
        stack.push(obj);
        obj = is.readObject();
      }

Replace the above fragments with single read/write operation:

write

      os.writeObject(stack);

read

      stack = (Stack) is.readObject();

This produces a different serialization - with objects shared via references. You can see this more easily from the structural view:

Exercises

Archive of project so far...

• https://github.com/wit-computing-msc-2017/pacemaker-console/releases/tag/lab03

Alternative Serializers

We have taken some time to abstract the serialization mechanism. This should enable us to introduce new serializers to adhere to alternative formats. Write two new serializers which support the following formats:

• Json
• Binary

Json is supported via xstream and the jettison library. A tutorial is available on the xstream site:

• http://x-stream.github.io/json-tutorial.html

A Binary serializer can be written just using the native streams in the JDK.

Features

Run your app, or the solution made available above and list all commands:

!la

This will list a range of 'built-in' commands + the commands we have implemented so far:

abbrev  name              params
---------------------------------------------------
cu      create-user      (first name, last name, email, password)
gu      get-user         (email)
gu      get-users        ()
du      delete-user      (email)
aa      add-activity     (user-id, type, location, distance)
al      add-location     (activity-id, latitude, longitude)

Review the command set as specified in the assignment:

abbrev    name                    params
---------------------------------------------------
lu    list-users          ()
cu    create-user         (first name, last name, email, password)
lu    list-user           (email)
lius  list-user           (id)
la    list-activities     (userid, sortBy: type, location, distance, date, duration)
la    list-activities     (user id)
du    delete-user         (id)
aa    add-activity        (user-id, type, location, distance, datetime, duration)
al    add-location        (activity-id, latitude, longitude)
cff   change-file-format  (file format: xml, json)
l     load                ()
s     store               ()

Some commands are more or less complete:

abbrev  name              params
---------------------------------------------------
cu      reate-user       (first name, last name, email, password)
gu      get-user         (email)
gu      get-users        ()
du      delete-user      (email)

Some are partially complete - the activities are in place, but do not include start time or duration:

abbrev  name              params
---------------------------------------------------
aa      add-activity      (user-id, type, location, distance)
al      add-location      (activity-id, latitude, longitude)

and these commands are yet to be tackled:

abbrev  name               params
---------------------------------------------------
lius    list-user          (id)
la      list-activities    (userid, sortBy: type, location, distance, date, duration)
la      list-activities    (user id)
du      delete-user        (id)
cff     change-file-format (file format: xml, json)
l       load               ()
s       store              ()

Prioritize these commands from simplest to most complex, and consider tackling the simpler commands first.