JS Includes: Object to File

Following up on the previous article about the Manifest module, here’s another very handy module that allows OL Connect Wiorkflow to quickly save any JavaScript object to a file, using the JS Includes functionality originally introduced in version 2023.1.

Creating JSON files

On many occasions, I find myself in need of saving values to a file in order to pass it on to another task or to OL Connect itself. That operation shouldn’t be overly complex, but after a few times doing it I realized that there are some pitfalls that I kept falling into. For instance, let’s say I want to save some parameters as an object. Here’s what I would usually do:

var serverPrefs = {ip:"127.0.0.1",name:"PROD"};
var fso  = new ActiveXObject("Scripting.FileSystemObject");
var file = fso.createTextFile("D:/ServerPrefs.json", true, false);
file.writeLine(JSON.stringify(serverPrefs));
file.close();

I think you’ll agree that there’s nothing particularly fancy about that code. It just works.

Or does it?

Well first of all, do you remember what the parameters to createTextFile() are? Oh, right, the documentation says the second parameter forces the command to overwrite any existing file. And there’s also a Unicode parameter that no one is quite sure what it does. Well I’ll tell you what it does: it puts a BOM at the start of the file (those two weird characters that identify some text files as using Unicode encoding). And trust me, you don’t want that.

Second, speaking of Unicode, the FileSystemObject is not your best friend when it comes to reading/writing files to disk. Unless your data always contains only ASCII characters, you will run into issues.

And third, the JSON file will not be properly formatted because most of us forget that the JSON.stringify() method allows you to specify additional parameters to properly format and indent the resulting JSON.

And of course, there’s the fact that you need to recreate that snippet in each script where the functionality is needed. Which usually involves you having to hunt down another instance of a script in which you implemented it, copy and paste it into your new code, fix the mistakes because you forgot to change the file name or something else, etc.

So yes, it works… but it’s still a bit of a pain in the behind if you have to do all of that on a regular basis.

Wouldn’t it be much simpler if we could just write something like this:

{ip:"127.0.0.1",name:"PROD"}.toJsonFile("D:/ServerPrefs.json");

Well, that’s exactly what our new Include module will allow us to do!

Improving the code

The first thing we’ll want to to before we even think of creating an Include module is to write out the cleanest, most efficient code we can and test it in an existing scripting task. So we’ll first create a function that receives an object as a parameter, as well as a file name, and then saves that object to that file as a JSON string:

var myObject = { 
   fName: "Andr\u00E9", 
   lName: "Ferré", 
   fullName: "Andr\xE9 'The Giant' Ferr%u00e9"
};

toJsonFile(myObject,Watch.GetJobFileName())

function toJsonFile(obj, filename) {
  var textStream   = new ActiveXObject("ADODB.Stream");
  var binaryStream = new ActiveXObject("ADODB.Stream");
  try {
    textStream.Type    = 2;          // 2=Text
    textStream.CharSet = "utf-8";
    textStream.Open();

    binaryStream.type  = 1;          // 1=Binary
    binaryStream.Open();

    var sObj = JSON.stringify(obj,null,2);
    // replace \uXXXX sequences with %uXXXX
    // replace \xXX sequences with %xXX
    // replace %uXX and %uXXXX sequences with actual character
    sObj = unescape( 
      sObj.replace( /(\\u)([0-9A-Fa-f]{4})/g, "%u$2" )
          .replace( /(\\x)([0-9A-Fa-f]{2})/g, "%x$2" )
    )

    textStream.writeText(sObj);
    textStream.position = 3
    textStream.CopyTo(binaryStream);

    binaryStream.saveToFile(filename,2);
  } finally {
    textStream.Close();
    binaryStream.Close();
  }
}

There’s a lot going on here. So let’s look at the most noteworthy items in that code.

First, you’ll notice that the object we want to save to file contains all kinds of weird and wonderful characters: \u00E9, %u00e9, %xE9, and é. Those all happen to represent the same character (é, aka &eacute in HTML), using various valid notations. Properly handling those notations is critical to making our method generic. We’ll get back to this in a moment.

Second, we have switched from using the FileSystemObject to the ADODB.Stream object. This gives us much better control over the encoding of the file. The stream object is kind of an in-memory file. You write to it just like you would a file, and you can move forward and backward within the stream. So the code creates a Text stream, sets its encoding to utf-8 (which is a requirement for JSON) and then writes the stringified version of our object to the stream, making sure to specify the proper formatting parameters so that the file is human readable (i.e. properly indented). But a UTF text stream automatically prepends a BOM sequence, which we don’t want in our final result. So after writing the object to the stream, the code resets the position of the stream to byte 3, in order to skip those pesky BOM characters.

The script also creates a second stream, in binary mode this time (which requires no encoding setting). It then copies the entire content of the text stream, starting from its current position (which explains why we positioned it at byte 3), to the binary stream, which is then saved to the file we specified as the second parameter to the function.

Note the use of the try ... finally construct around the entire code to ensure that, whatever happens, both streams are closed properly.

This flavor of the code is much more robust than our previous snippet, but it’s obviously more complex. And that’s where wrapping it into a reusable Include file will come in handy.

Handling escape sequences in strings

Between the JSON.stringify() and the textStream.writeText() methods, the code uses the unescape() method to replace escape sequences with actual characters. And while you’d think that would be enough, well… it isn’t. You see, unescape() only handles the %uXXXX or %xXX notations, but not the fairly common \uXXXX or \xXX ones. That’s why the code uses a couple of regular expressions to convert the latter notations to the former, which allows unescape() to do its job properly.

A note to our American friends: encoding issues are widely unknown – or ignored – in America because accented characters are relatively rare in common lexical elements. But if you want to properly handle surnames or geographic locations (or Spanish characters as in El Niño), then you must make sure that your code can handle those encoding issues.

Changing the Object prototype

Okay, so now we have a working function and we could stop there. But ideally, we’d want to be able to save any JS object to a JSON file, not just objects we create on the fly. One of the nice perks of JavaScript is that you can extend any object’s functionality by modifying its prototype (think of the prototype as a blueprint for any object and its descendants). And since every single object in JavaScript descends from the Object object, well then modifying the Object‘s prototype would impact all JS objects.

So let’s modify the above code in order to add the toJsonToFile() method to all JS objects.

var myObject = { name: "Me", id: 12345};
myObject.toJsonFile("D:/myObject.json")

Object.prototype.toJsonFile = function(filename) {
  var textStream   = new ActiveXObject("ADODB.Stream");
  var binaryStream = new ActiveXObject("ADODB.Stream");
  try {
    textStream.Type    = 2;          // 2=Text
    textStream.CharSet = "utf-8";
    textStream.Open();

    binaryStream.type  = 1;          // 1=Binary
    binaryStream.Open();

    var sObj = JSON.stringify(this,null,2);
    sObj = unescape( 
      sObj.replace( /(\\u)([0-9A-Fa-f]{4})/g, "%u$2" )
          .replace( /(\\x)([0-9A-Fa-f]{2})/g, "%x$2" )
    )
    textStream.writeText(sObj);
    textStream.position = 3
    textStream.CopyTo(binaryStream);

    binaryStream.saveToFile(filename,2);
  } finally {
    textStream.Close();
    binaryStream.Close();
  }
}

The code is almost identical, but instead of creating a global function, it adds it to the Object.prototype as a new method. The method no longer receives an object parameter because, by being part of the object prototype, it knows how to access its owner object through the this keyword.

Now to use the method, you don’t even have to first assign the object to a variable, you can save it directly, as with this array:

[1,2,3,4,5,6,7,8,9].toJsonFile("D:/myArray.json")

But of course, this new Object method is – for now – only added to the Object prototype for the duration of this script. Once the script has completed, the change to the Object prototype is discarded. So let’s now add the final piece: using an Include file to permanently add the method to the Object prototype.

The ObjectToFile module

Now that we’ve tested the code inside a standalone script, it’s time to turn it into a reusable module. That part is deceptively easy. You simply need to copy the entire code for the method and wrap it inside an immediate function that the JS Engine executes each time it starts:

(function() {

  Object.prototype.toJsonFile = function(filename) {
    /*
      ... same content as the last code snippet, above
    */
  }

}());

So really, our earlier code is simply inserted inside the curly brackets of a ( function(){}() ) statement, which is JavaScript’s somewhat convoluted way of declaring and immediately executing code.

Save that file to something like ObjectToFile.js , and insert the full path name of that file in your C:/ProgramData/Objectif Lune/PlanetPress Workflow 8/PlanetPress Watch/includes.json file.

Now, each time a scripting task is run, any JS object can take advantage of its “native” toJsonFile() method. Oh, and you don’t even need to stop/restart the Workflow service: the changes are effective immediately.

Caveat

This module is so handy that I hesitated before mentioning the only caveat I have found when using it. But I figured I might as well discuss it immediately, even though it is a bit technical and highly unlikely to ever pose a problem. And besides, there is an easy workaround.

The issue is that after we add the toJsonFile() method to the Object prototype, the method name is listed as part of the enumerable members of each object. So for instance, if you were to have a standard object whose properties you want to list, you’d use something like this:

var myObject = { name: "Me", id: 12345};
for (var p in myObject){
  Watch.log(p,2)
}

and this would give you the following output:

name
id

But once the toJsonFile() method has been added to the Object prototype, you get the following result:

toJsonFile
name
id

In theory, the JavaScript standard does allow us to add the toJsonFile() method as a non-enumerable member of the Object class, using the native defineProperty() method, but unfortunately, Microsoft JScript’s implementation of that method is defective. And believe me, I’ve tried… So we have to live with the fact that the newly added method is enumerable. Fortunately, if you ever need to list the members of an object without any of its methods, you can easily work around the issue with something like this:

for (var p in myObject){
  if(typeof myObject[p] === "function") continue;
  Watch.log(p,2)
}

Wrapping it up

This module is possibly the most handy in the arsenal of Includes I have built over the last few months. In fact, it is so handy that I also added a fromJsonFile() method to the Object prototype, which allows me to create a JavaScript object from a JSON file using some pretty human-readable syntax:

var myObj = Object.fromJsonFile("D:/somefile.json");
// or ...
var myObj = {}.fromJsonFile("D:/somefile.json");
// or even ...
var myObj = "".fromJsonFile("D:/somefile.json");

The latter 2 examples are not necessarily clearer than the first one, but they illustrate how both fromJsonFile()/toJsonFile() are now methods of every single type of object in JavaScript once we have included our module.

Let me know in the comments if you find this module (or even this article!) useful.

_{(Top image courtesy of Json file icons created by juicy_fish – Flaticon)}