JSON Tools for Pascal

I decided to write a small yet capable JSON parser and builder. A JSON parser converts plain text in Javascript object notation format into a set of nodes which can be queried. A JSON builder allows you to dynamically create and manipulate nodes, then output those nodes as valid JSON text.

Update

An update to JsonTools has been posted to its github repository. This update fixes a problem with escaped double quotes "\"" and adds a few conveniences. The most significant convenience is the addition of the Force method.

The Force method allows users to add or modify a chain of objects, even if none exists. Force will search a path given, and return a node. If any part of the path doesn't exist, it will be created for you.

{ Force a series of nodes to exist and return the end node }

function Force(const Path: string): TJsonNode;

Consider the following example:

show source     
var
  N, C: TJsonNode;
begin
  N := TJsonNode.Create;
  N.Force('customer/first').AsString := 'James';
  N.Force('customer/last').AsString := 'Milligan';
  N.Force('address/street').AsString := '123 Skippy Lane';
  N.Force('address/state').AsString := 'FL';
  C :=  N.Force('cart').Add;
  C.Force('name').AsString := 'shorts';
  C.Force('price').AsNumber := 25.12;
  N.Force('cart').Add.Add('name', 'gloves').Parent.Add('price', 30);
  N.Force('total').AsNumber := 55.12;
  WriteLn(N.Value);
  N.Free;
end;

This will result in the following output:

{
	"customer": {
		"first": "James",
		"last": "Milligan"
	},
	"address": {
		"street": "123 Skippy Lane",
		"state": "FL"
	},
	"cart": [
		{
			"name": "shorts",
			"price": 25.12
		},
		{
			"name": "gloves",
			"price": 30
		}
	],
	"total": 55.12
}

Another addition is the Exists method. Exist will check a path and return True if a node is found matching the given path.

{ Search for a node using a path string and return true if exists }

function Exists(const Path: string): Boolean;

A parameterless Add method overload has been added. This overload of Add converts the node to an array, if it's not already, and returns a new empty object node belonging to the array.

{ Convert to an array and add an item }

function Add: TJsonNode; overload;

Finally, a Find method overload has been added that places the found node in an output parameter, and returns True if a node matching the path was found.

{ Search for a node using a path string and return true if exists }

function Find(const Path: string; out Node: TJsonNode): Boolean; overload;

Why use JSON?

JSON is a standardized text data format that has increasingly becoming ubiquitous. It works on any computer system, its portable, its concise, and its human readable and editable. Many web based API calls require either JSON content as post data, or return JSON content as results. Many desktop systems are switching to JSON as a file format for configurations, settings, layouts, or other data persisted to disk.

Why write a JSON parser when one already exists?

The reason we wrote this parser and builder comes down to two primary reasons.

we wanted a very simple interface to work with JSON in Pascal that works in the manner to which we are accustomed. More on this below.
We wanted to write a fairly small yet complete solution cleanly implemented in one class with no dependencies other than the SysUtils and Classes units.

How does our JSON parser make working with JSON easier?

Our parser is very easy to use. Simply add the JsonTools unit to your uses clause, then you can parse a valid JSON document like so:

N := TJsonNode.Create;

N.Parse(YourJsonText);

Where you are done with your TJsonNode simply write Free. You only need to free a TJsonNode if you created it directly.

N.Free;

In addition to calling Parse you can also use these methods to parse a JSON document.

N.Value := YourJsonText;

Or if the JSON is in a file you might write:

N.LoadFromFile(YourJsonFileName);

These methods do the same thing as Parse. They build a tree of JSON nodes by parsing JSON data. To get your JSON text data back out of any node you can simply write:

WriteLn(N.Value);

And the nodes are converted back into a nicely formatted JSON document. If you wanted to print out each node, then the entire document you might write:

N.Value := '[ "Hello", "World!" ]';

for C in N do

WriteLn(C.Value);

WriteLn(N.Value);

Which would result in the following output:

"Hello"
"World!"
[
    "Hello",
    "World!"
]

If you are unsure if your source JSON text is valid, you can write this bit of code to check if it is correct:

show source     
S := '{ this is not valid json }';
if N.TryParse(S) then
  // Do something with your data
else
  // Your input did not contain valid JSON text

More things you can do with JSON

Now that we know how this JSON object can parse text, let's look at an example of how it can navigate and manipulate JSON nodes. Here is a bit of JSON text data we'll be using in a few examples to follow.

show source     
{
    "name": "Alice Brown",
    "sku": "54321",
    "price": 199.95,
    "shipTo": {
        "name": "Bob Brown",
        "address": "456 Oak Lane",
        "city": "Pretendville",
        "state": "HI",
        "zip": "98999"
    },
    "billTo": {
        "name": "Alice Brown",
        "address": "456 Oak Lane",
        "city": "Pretendville",
        "state": "HI",
        "zip": "98999"
    }
}

If this data were in a file named orders.json, we could use it like so:

N.LoadFromFile('orders.json');

WriteLn('The price is ', N.Find('price').Value);

WriteLn('The customer is ', N.Find('billTo/name').Value);

The output of which would be:

The price is 199.95
The customer is "Alice Brown"

If you wanted to change the shipTo to match the billTo, you could simply write:

N.Find('shipTo').Value := N.Find('billTo').Value;

And all the nodes under billTo will be copied over the shipTo. What this means is that my object can parse text on the node level, allowing you to compose a JSON document from several sources or move and copy data easily. You could also load and save various child nodes from separate files or streams. How cool is that?

If you just want to change the name of the customer you could write:

N.Find('billTo/name').AsString := 'Joe Schmoe';

Notice we are using AsString above instead of Value. This is due to the nature of the Value property. Whenever you set Value the current node will try to parse the incoming value as a JSON fragment. The input value of 'Joe Schmoe' is not valid JSON. To let our object know we intend to set a string value we can use the AsString property.

If you do not want to use AsString would need to write this statement instead, which makes 'Joe Schmoe' a valid JSON string:

N.Find('billTo/name').Value := '"Joe Schmoe"';

Notice we've enclosed out new name in " double quote characters. Double quotes are required to create JSON strings. Since we've added double quote characters the second statement now feeds a valid value to our Value property.

Here are some examples of valid values you can use when setting the Value property of a node.

show source     
N.Value := 'true';   // node becomes a bool node
N.Value := '3.1415'; // node becomes a number node
N.Value := 'null';   // node becomes a null node
N.Value := '[]';     // node becomes an array
N.Value := '{}';     // node becomes an object

In the examples above N is changed to a different kind of node with each statement. If N is the root node, the first three statements will fail, as a root node is required to be either an object or array.

Alternately, the five statements below do the exact same thing as the five statements above, but with a bit of added type safety:

show source     
N.AsBoolean := True;
N.AsNumber := 3.1415;
N.AsNull;
N.AsArray;
N.AsObject;

Note that AsNull, AsArray, and AsObject do not have an := assignment operator. This is because the Value of each of these types is fixed. A null node is null, an array node is array, and an object node is object. These three statements have the effect of converting the node kind and resetting its value.

Switching to another node

As noted in the last section, attempting to set the root node to a value that is not an object or an array is an error. The JSON standard dictates that the root level of all JSON objects must either be an object or an array

To switch to another node you might write:

N := N.Find('shipTo/name');

N.Add('first', 'Joe');

N.Add('last', 'Schmoe');

If the shipTo/name was a string kind, it will switch to an object kind when a node is added. After temporarily switching nodes, to get a reference back to our root we could type:

WriteLn(N.Value);

N := N.Root;

And now we can be sure N refers to the root node again. The output of above snippet would have been:

{
    "first": "Joe",
    "last": "Schmoe"
}

Adding and removing nodes

Nodes can be added or removed programatically using special purpose methods. If you want to add a value to an object (or an array) you can write:

N.Add('note', 'Customer requested we change the ship to address');

This would add the following to our JSON object:

"note": "Customer requested we change the ship to address"

In our pascal code, if N is an array then the first parameter will be ignored as arrays in JSON do not have named values for their child nodes.

Also notice when we use the Add method we do not need to use JSON strings. We can just use normal strings and they will be converted to JSON format internally. This internal formatting allows us to add strings with multiple lines or other format encoding, and internally they will be retained as JSON compatible strings.

Add is overloaded allow you to add bool, number, string, and more:

N.Add('discount', -8.50); // adds a number node

N.Add('verified', True); // adds a bool node

N.Add('wishlist', nkArray); // adds an array node

A particular feature of JSON is that it only allows one node of a given name at specific level. This means you Add the same name again, then the existing node will be returned, and its value will be changed to match the second argument of Add.

A child node can be removed either using the Delete method or Clear method.

N.Delete('note'); // removes the note node

N.Delete('shipTo'); // removes the shipTo node

To delete items in an array, simply use the string representation of the array index:

N.ToArray.Add('note', 'Please call the customer');

N.Delete('0');

In the example above our N is converted to an array and an item is added. Since arrays do not use names to index their child nodes, the first argument 'note' is discarded. The second line in the example removes the 0 (arrays are 0 based) item. If N was not as array before the first line line, then its values are discarded and it would only contain our string "Please call the customer".

To remove all child nodes of an object or array, simply type:

N.Clear;

This will remove every node below the current node. Clear has no effect on null, bool, number, or string nodes.

The basic properties of every JSON node

Every node at its core stores the following information:

show source     
property Root: TJsonNode     // the root of your JSON document (read only)
property Parent: TJsonNode   // the parent of your JSON node (read only)
property Kind: TJsonNodeKind // the kind of node, such as object, array, string etc (read write)
property Name: string        // the name of node (read write)
property Value: string       // the value of the node in JSON form (read write)

The Kind property uses an enumerated type defined as one of these possible values:

nkObject, nkArray, nkBool, nkNull, nkNumber, nkString

When Kind is set to a different than current value, the node is reset. For object and array kinds this means all children are removed. For bool, null, number, and string kinds the reset values are false, null, 0, and "" respectively.

If a duplicate Name is set, then the existing node of the same name is removed and replaced by the current node. You can only set the name of a node if the parent is an object, otherwise any attempts to change the name are ignored.

When Value is set, internally the value you pass is parsed as JSON and the entire value portion of the node is replaced, potentially changing the node kind in the process. It is important to note that the root node value must be a valid JSON object or array. It cannot be any other kind.

Here are a few examples of how the Value property can be used.

N.Child('sku').Value := '[ "54321", "98765" ]'; // sku becomes an array of values

N.Find('shipTo/name').Value := '{ "id": 5028, "primary": true }'; // name becomes object

In addition to being able to use the for...in enumerator on any node, you can also use these three properties to enumerate child nodes:

function Child(Index: Integer): TJsonNode; // get a child node by index

function Child(const Name: string): TJsonNode; // get a child node by name

property Count: Integer; // return the number of child nodes

Note that Child is not the same as the previously mentioned Find method. The Find method searches a path, while the Child method looks for an exact name match directly under the current node.

Source code

All the source code to this parse if freely available under the GPLv3 license. It's hosted on github here and the interface source is listed here for your reference. Feel free to send me your comments or feedback.

show source     
type
  TJsonNode = class
  public
    { A parent node owns all children. Only destroy a node if it has no parent.
      To destroy a child node use Delete or Clear methods instead. }
    destructor Destroy; override;
    { GetEnumerator adds 'for ... in' statement support }
    function GetEnumerator: TJsonNodeEnumerator;
    { Loading and saving methods }
    procedure LoadFromStream(Stream: TStream);
    procedure SaveToStream(Stream: TStream);
    procedure LoadFromFile(const FileName: string);
    procedure SaveToFile(const FileName: string);
    { Convert a json string into a value or a collection of nodes. If the
      current node is root then the json must be an array or object. }
    procedure Parse(const Json: string);
    { The same as Parse, but returns true if no exception is caught }
    function TryParse(const Json: string): Boolean;
    { Add a child node by node kind. If the current node is an array then the
      name parameter will be discarded. If the current node is not an array or
      object the Add methods will convert the node to an object and discard
      its current value.
 
      Note: If the current node is an object then adding an existing name will
      overwrite the matching child node instead of adding. }
    function Add(const Name: string; K: TJsonNodeKind = nkObject): TJsonNode; overload;
    function Add(const Name: string; B: Boolean): TJsonNode; overload;
    function Add(const Name: string; const N: Double): TJsonNode; overload;
    function Add(const Name: string; const S: string): TJsonNode; overload;
    { Convert to an array and add an item }
    function Add: TJsonNode; overload;
    { Delete a child node by index or name }
    procedure Delete(Index: Integer); overload;
    procedure Delete(const Name: string); overload;
    { Remove all child nodes }
    procedure Clear;
    { Get a child node by index. EJsonException is raised if node is not an
      array or object or if the index is out of bounds.
 
      See also: Count }
    function Child(Index: Integer): TJsonNode; overload;
    { Get a child node by name. If no node is found nil will be returned. }
    function Child(const Name: string): TJsonNode; overload;
    { Search for a node using a path string and return true if exists }
    function Exists(const Path: string): Boolean;
    { Search for a node using a path string }
    function Find(const Path: string): TJsonNode; overload;
    { Search for a node using a path string and return true if exists }
    function Find(const Path: string; out Node: TJsonNode): Boolean; overload;
    { Force a series of nodes to exist and return the end node }
    function Force(const Path: string): TJsonNode;
    { Format the node and all its children as json }
    function ToString: string; override;
    { Root node is read only. A node the root when it has no parent. }
    property Root: TJsonNode read GetRoot;
    { Parent node is read only }
    property Parent: TJsonNode read FParent;
    { Kind can also be changed using the As methods.
 
      Note: Changes to Kind cause Value to be reset to a default value. }
    property Kind: TJsonNodeKind read FKind write SetKind;
    { Name is unique within the scope }
    property Name: string read GetName write SetName;
    { Value of the node in json e.g. '[]', '"hello\nworld!"', 'true', or '1.23e2' }
    property Value: string read GetValue write Parse;
    { The number of child nodes. If node is not an object or array this
      property will return 0. }
    property Count: Integer read GetCount;
    { AsJson is the more efficient version of Value. Text returned from AsJson
      is the most compact representation of the node in json form.
 
      Note: If you are writing a services to transmit or receive json data then
      use AsJson. If you want friendly human readable text use Value. }
    property AsJson: string read GetAsJson write Parse;
    { Convert the node to an array }
    property AsArray: TJsonNode read GetAsArray;
    { Convert the node to an object }
    property AsObject: TJsonNode read GetAsObject;
    { Convert the node to null }
    property AsNull: TJsonNode read GetAsNull;
    { Convert the node to a bool }
    property AsBoolean: Boolean read GetAsBoolean write SetAsBoolean;
    { Convert the node to a string }
    property AsString: string read GetAsString write SetAsString;
    { Convert the node to a number }
    property AsNumber: Double read GetAsNumber write SetAsNumber;
  end;

Comparisons

We've added a new page to test our our new parser in comparison to several other pascal based parsers. You can read more about these tests here.

The results show that our parser is dramatically better than the other parsers we were able to find and test. Better both in terms of speed and correctness. Though we believe our test are fair, they are by no means exhaustive, some please be mindful of that fact when considering our results.