Tag Archives: JSON

JSON-related posts

Accessing Google APIs using Restty

Posted on by
Reply

Starting with version 0.22, Restty includes authentication module restty-oauth2 supporting OAuth 2.0 authentication protocol utilized by Google in theirs APIs. Even better, Restty also includes full set of Active JSON templates for accessing Google Tasks and URL Shortener APIs (see gtasks and gshort profiles).

In order to make oauth2 module available for Active JSON templates, you need to register it using _config.js file for your profile.

Enabling OAuth module in Restty config

1. Enabling OAuth module in Restty config

Section contextObjects of the config file is used to register Context Objects that will be available in Active JSON templates. Context Objects have to be defined as key – value pairs. Key determines the name of the object and the value provides required information for the object to be created. It may be either a string value (in this case it has to define a java class name for the object implementation, or it may be an object (in this case it must have an parameter implementation that has to provide the java class name). Additionally, there may be a parameter settings that can be used to configure that Context Object. In our case, it has a single configuration parameter useCache which is set to true. This parameter determines whether the oauth module has to persist received authentication information in a file. The information is stored in the plain text file, so be careful, with this option. However, if you disable the cache, you will be asked to grant access to you data every time you run Restty command that requires authentication, which is a bit inconvenient. Remember, that you may always go to .\work directory and remove files that contain sensitive information (the files have names starting with restty-oauth2_).

Section settings is optional in the Context Object configuration. If omitted,  the default settings will be applied. As for useCache default value is true, instead of using complex object to describe $googleAuth Context Object we could have used the same short notation as we used to define $args.

For the commands that require authenticated access,  all you need to do is add parameter Authorization to the header description.

Providing authorization token to Google API

2. Providing authorization token to Google API

As you can see, we assigned string value “Bearer ” + authorization token returned by $googleAuth to the HTTP header parameter Authorization. This code normally goes to _prototype.js, so you don’t need to include it into the individual command templates.

Method getToken, receives two arguments:

  • scope – this is an authentication scope specific to the current Google API. This argument may be either string of array of string when several scopes are used. For the full set of available scopes, please, refer to Google API reference.
  • tag – this argument is used to cache authentication information. Information for every tag is stored separately. You have to use different tags for different scopes. Also, using different tags allows accessing different accounts – all you need to do is use a separate tag for each account.
Multiple accounts support

3. Multiple accounts support

When there is no cached authentication information available or you disabled the caching, you will be asked to grant Restty access to the specific Google API.

Grant Access Confirmation

4. Grant Access Confirmation

For the Restty command to succeed, you have to press “Allow access” button. This will provide Restty with the authorization tokens to perform trusted API calls.

Google URL Shortener - Access granted

5. Google URL Shortener - Access granted

In the case, when a command has many input arguments (especially, when many of them are optional), it would be inconvenient to pass values sequentially one-by-one, so they can be passed by name like shown on the following screenshot for insert-task command utilizing Google Tasks API.

Google Tasks API - passing arguments by name

6. Google Tasks API - passing arguments by name

Using Restty is easy – download it and see for yourself.

For your reference, here is a complete list of Google URL Shortener commands implemented in Restty with the arguments:

  • get - returns information about single url
    • url – required – short url for lookup
    • fields – optional, default = “short” determines, whether to show additional information for the specified url (allowed values: [short | all]). For the “short” option it returns original url value and clicks analytics - number of clicks for the url in specific intervals (two hours, day, week, …). For the “all” it returns all available data including analytics
  • list - returns list of all urls associated with user’s account. The number of urls returned, is limited by 30 items. If there are more urls available, pageToken argument may be used to navigate through the result.
    • pageToken – optional - can be used to navigate through the result when there are more then 30 items available.
    • fields - optional, default = “short” determines, whether to show additional information for the specified url (allowed values: [short | all]). For the “short” option it returns original and short url values. For the “all” it returns all available data including analytics
  • new - creates new short url from the provided original url
    • url – required – original url

And for Google Tasks API:

  • get-list – returns detailed information about a single Task List, if id is provided, otherwise returns list of available Task Lists.
    • id – optional – Task List id
    • maxResults – optional, default = 20 – max results returned per page
    • pageToken – optional - can be used to navigate through the result when there are more then maxResults items available
  • insert-list - creates a new Task List with the given title
    • title – required – title of the Task List
  • update-list - updates an existing Task List title
    • id – required – Task List id to be updated
    • title – required – new title
  • clear-list - clears (hides completed tasks) specified Task List
    • list – required – Task List id
  • delete-list - deletes specified Task List
    • id - required - Task List id
  • get-task - returns detailed information about a single Task, if id is provided, otherwise returns list of available Tasks
    • id – optional – Task id
    • list – optional, default = “@default” – Task List id
    • showCompleted – optional, default = false - flag indicating whether to show completed Tasks
    • maxResults – optional, default = 20 - max results returned per page
    • pageToken – optional - can be used to navigate through the result when there are more then maxResults items available
    • updatedMin - optional  - lower bound for Tasks update date in RFC 3339 format (for example 2012-05-28T12:00:00).
    • completedMax - optional  - higher bound for Tasks completed date in RFC 3339 format
    • completedMin - optional  - lower bound for Tasks completed date in RFC 3339 format
    • dueMax - optional - higher bound for Tasks due date in RFC 3339 format
    • dueMin - optional - lower bound for Tasks due date in RFC 3339 format
    • showDeleted - optional, default = false - flag indicating whether to show deleted Tasks
    • showHidden - optional, default = false - flag indicating whether to show hidden Tasks
    • fields - optional, default = “short” - determines level of details to request. Allowed values [min | short | all]. Min returns only titles; short - id, title, status, due; all – returns all fields
  • insert-task- creates a new Task
    • list - optional, default = “@default” - Task List id
    • parent - optional – parent Task id – to create a sub-Task
    • previous - optional - previous Task id – to insert a Task into specified position
    • title - required - title of the new Task
    • notes - optional - description of the new Task
    • due - optional - due date of the new Task in RFC 3339 format
    • completed - optional, default = false – flag showing whether a Task is completed
  • move-task- moves a Task to a new position
    • id – required – Task id
    • list – optional, default = “@default” – Task List id
    • parent - optional - parent Task id – to make the Task a sub -Task
    • previous - optional - previous Task id – to move the Task into specified position
  • update-task- updates an existing Task
    • id - required - Task id
    • list - optional, default = “@default” - Task List id
    • title – optional - new title of the the Task
    • notes - optional - new description of the Task
    • due - optional - new due date of the Task in RFC 3339 format
    • completed - optional - flag showing whether the Task is completed
  • delete-task- deletes an existing Task
    • id - required - Task id
    • list - optional, default = “@default” - Task List id

 

Using Restty for testing JSON-based Web Services

Posted on by
Reply

As a Software developer, I often have to test, explore or script different Web Services. Whether it’s a service I or some of my colleagues work on or it’s a 3d party service we use – doesn’t matter. The reality is that quite often we don’t have a proper client to test the service with, or the client is out of date and doesn’t support some of the interfaces. Whatever the reason is, I quite often ended up using either wget and curl to fire HTTP requests to my services, or using browser-based tools like REST Console (it’s a Google Chrome plugin).

However, using generic HTTP client tools require some tweaking and is a distraction from the main task. On the other hand using REST Console and similar GUI tools is only efficient when you need to test some particular request quickly and just once. If you want to script some tasks or schedule it using cron or just want to prepare some template for repetitive use, those GUI tools have little to offer. There is, of course, another way – to develop a custom command line client specific for your Web Service, but obviously, it’s the most time-consuming approach.

Or, I would say, it was the most time-consuming approach, before we developed Restty.

Restty is a template based universal client for JSON-based Web Services. It combines power of Apache HTTPClient and Mozilla Rhino JavaScript engine with a concept that we call Active JSON. In order to adapt it to some specific Web Service, all you need to do is prepare a set of templates using JSON and a bit of JavaScript. Restty is an Open Source project under Apache 2.0 license.

Restty processes commands in several steps. First step is Template Lookup. Based on restty.profile parameter and first command line argument supplied to the script, it loads appropriate Active JSON template files (_prototype.js is always loaded – it’s used to determine default values).

Template Lookup

1. Template Lookup

Second step is Template Compilation. During this step two templates determined on the previous step are being combined into single template and then result is compiled into JavaScript object.

Template Compilation

2. Template Compilation

When combining two templates, every element is merged using specific rule. As a result of this step we get compiled template – a JavaScript object representing our template.

The next stage is Command Building. During this step input arguments and corresponding values are determined. Then these arguments are made accessible within the template using Context Object $args. And finally, a command is created from the template. All constant values (url, method, …) directly transferred from the template to the command, all functions (body) are executed and the result they returned is assigned to the corresponding field of the command. As the result of this step, we get a command, that contains all the information required to fire an HTTP request towards our Web Service.

Command Building

3. Command Building

Please note, that the arguments may be provided to the command either sequentially, as shown in this example, or as name-value pairs. In the latter case the user input would look like

This comes useful when there are more then 2-3 arguments defined for a command. Also, it helps skipping optional arguments.

The next step is Command Execution – using our command object, we perform HTTP request. Returned result we pass to the command for future use.

Command Execution

4. Command Execution

The last stage is Result Processing (formatting and output). Currently, Restty supports only JSON output format, so the formatting really means that the JSON response is “butified”. The result of formatting then is output to the console.

Result Processing

5. Result Processing

In the conclusion let me highlight the steps that need to be taken in order to adapt Restty to your specific Web Service:

  1. Prepare start file (.bat or .sh) with the appropriate meaningful but short name. Place it in the Restty root directory. To keep things simple, use the same name for Restty profile (restty.profile parameter in your start file).
  2. Create profile directory .\config\profile\[your profile name] or, even better, use one of the provided template directories, just rename it to your profile name. Your profile directory must contain files _config.js – configuration file and _prototype.js - template prototype file.
  3. In your profile directory, create one template file for each command you want to define. Template file names have to follow this format: [one word command name not starting with "_"].js

We hope, you will find Restty a useful tool and if you do, please, let us know. Also, contact us, if you have some ideas how to improve it or want to contribute to the project.

Active JSON concept

Posted on by
2

Well, these days it seems that everyone knows what JSON is and what it’s for. But, let me recap anyway.

JSON is a lightweight human-readable data-interchange format, a popular alternative to XML these days. So, why is it so popular? I believe, the main reason, is that it’s very simple – JSON is a subset of JavaScript (a tiny one), however it offers everything what’s required to properly represent structured data – it has strings, numbers and booleans to represent scalar data types, it has objects to represent sets of key-value pairs, it has arrays to represent vector data types and, of course, it allows combining all of those – Figure 1. Looks nice and complete, doesn’t it?

Plain JSON

Figure 1 - Plain JSON example

Like I mentioned before,  JSON is a subset of JavaScript. So, what if we borrow one more data type from JavaScript – function. What, if we also agreed that our function must return some value which, in turn must be representable by JSON (consist of JSON-supported types). In this case our JSON could look like this:

Active JSON

Figure 2 - Active JSON example

As you probably noticed, we also stripped object keys form quotation marks wherever we could, to make code look even cleaner (JavaScript allows that if a key value is a valid JavaScript identifier)

What we got is something in between plain vanilla JSON and full-featured JavaScript.

Now, how do we process it? We may use some JavaScript engine (Mozilla Rhino for example) to compile such JSON in JavaScript object, then we execute the corresponding function, and finally, substitute the function with the result it returned. This process may be represented with following JavaScript code Figure 3.

JavaScript representation of Active JSON processing

Figure 3 - JavaScript representation of Active JSON processing

Now variable json points to JavaScript object equivalent of JSON displayed on Figure 1. All we have to do is to convert it back into JSON format.

I call this concept – Active JSON. It’s just a bit more complex then normal JSON, but a lot more dynamic. What’s the use of it? It’s not particularly useful for data transfer or persistence, I must admit, however it may be used to define JSON templates. All you need to do, is provide some context, while executing the dynamic part like shown on Figure 4.

Active JSON Processing

Figure 4 - Active JSON Processing

You might say: I could use any scripting language (JavaScript, php, Groovy, …) to generate JSON and get similar result. This is true, however, you would hardly get cleaner, simpler and shorter template code then with Active JSON.

If you want to see Active JSON in action, you may check out Restty - tool for accessing RESTful JSON-based Web Services that we built around it.