Data Sources

From Memento Database Wiki
Revision as of 16:32, 5 March 2017 by UnConnoisseur (talk | contribs)
Jump to navigation Jump to search
« Page as of 2017-03-05, editions Mobile 4.3.0, Desktop 1.1.0 »

The Autofill facility in Memento allows users to look up information from data sources on the Web, in Memento libraries, or from other data sources. Certain Web data sources are included by default in the Memento application for the convenience of users. For other lookups, a custom data source is needed.

A Custom Data Source is one in which a script acts as a source of data to fill fields with data from any service that has an API, from other libraries, or from other sources.

Lifecycle of an Autofill from a custom data source

  1. When adding an entry, the user types a search key into a field in the Entry Edit card.
  2. The data source script is then executed, with the search key passed in as the global variable query.
  3. The user is presented with a list of matching objects found by the script.
  4. The user chooses one of the objects, and its properties are written to the mapped entry fields according to data load rules.
  5. If the script has also returned a function, that script function is then called with the selected object's id property as an argument, and when the script function returns the object, now loaded with additional data, the object's new properties are loaded into mapped fields according to data load rules.

Create a custom data source

Step 1 — Add a new custom data source for Autofill

  1. Open the Library Edit screen, click on the Autofill tab, and then click +.
  2. Select the type of search — By Name or By Barcode.
  3. Select Custom source.
  4. Select the field to be used for data search within the source.
  5. You are presented with an Autofill Data Load Rules edit card. For each field to be loaded from the data source, enter the object property name that will identify that field. Then, select the field to be loaded from that property.

Step 2 — Write the script

The script is executed in the context of either a library or an entry.

It can use the same functions and objects as a trigger.
A script has the same security parameters as a trigger.

The result of the script should be returned as an array object.

See #Script Details below for details on script writing.

Step 3 — Set Autofill rules (Map data to fields)

After the script is written, it is necessary to map the properties of the returned object to the fields of the library:

  1. Close the script edit card and click +.
  2. This will open the card for creation of autofill rules.
  3. Identify the object property returned by the script.
  4. Select the library field to receive the data.
  5. Create rules for the remaining object properties.

Script details

Initial search results data load

Many services (Web APIs) return general information without details as a result of a search. This is analogous to the search results page of a Google or other Web search. This information hopefully gives the user enough information to make the decision to choose a single result from among others.

In this case, the properties of the source object are:

  • The title, or name, of the item matching the search key.
  • A brief desc, or description, of the item for further clarification.
  • A thumb, or image thumbnail, can help the user to make that decision.
  • If more fields are wanted from the source, a function in the script will be called to get them, and an id property is then needed to identify to the function the result chosen by the user.

Additional data load

To extract complete detailed information, an additional API request is needed. In this case, a function for additional data load should be used. The argument for this function is the identifier of the object selected by the user. The function returns an object whose properties will be loaded into the fields of the entry according to Autofill rules.

As an example, let's consider a Web service with an API with two methods:

  1. http://api.example.com/search?q='search query' — This call does a search for an object and returns only identifiers and the names of the items. The result is an array of JSON objects. Each object has title and id properties.
  2. http://api.example.com/get?id=123 — This call fetches detailed information from the item based on the id. The result is a JSON object.

Here's an example of a script requesting data from a generic Web service:

var jsonResult =
   http().get("http://api.example.com/search?q=" +
   encodeURIComponent(query));
result (
   JSON.parse(jsonResult), 
   function(id) {
      return http().get("http://api.example.com/get?id=" + id);
      });

Data source API

Data source scripts have access to the Memento JavaScript Library. Data Sources have their own additional API, as follows.

result(objects, getDetailsFunction)
Function to return the initial result of the script — an array of objects representing the search results and a function to be called for details.
Arguments
objects
List of objects which will be presented to the user for selection
getDetailsFunction
Optional argument — function to fetch detailed data. This function is called when one of the objects is selected.
Source object
Objects sent to the result function can have the following standard properties. When present, these properties will be displayed to the user for selection:
Properties
title
Name of the object (required)
desc
Additional text shown under the name of the object (may be helpful to the user)
thumb
Reference to the image of the object (may be helpful to the user)
id
Object identifier (required for additional data). Used if result() is called with a second argument.

Examples

Example — Filling data from another library

Say we have a library Organizations, and now we are building a new library that has organization information as a part of it. When we add an entry, we want to provide a search key and have the rest of the organization fields filled from the Organizations library. This is different from referencing the Organizations library, as with a Link to Entry field and Calculation or JavaScript fields; in the case of Autofill, the data is copied from the Organizations library as a separate copy.

So, we edit the new library, go to the Autofill tab, and press the + button to add an Autofill rule.

var anotherLib = libByName(AnotherLib)
var entries = anotherLib.find(query);
var resultArray = [];
for(var i in entries ) {
	var object = {};
	object[title] = entries[i].title;
	object[desc] = entries[i].description;
	object[number] = entries[i].field(Number);
	resultArray.push(object);
}
result(resultArray);

Example — Filling data from a Web data service

Let's consider service http://www.discogs.com/ — Provides information about music albums. This service has an API documented at http://www.discogs.com/developers.

An example of a JavaScript library that issues a request to this service is available in Memento's script repositories. To connect this library:

  1. Open custom data source editor
  2. Click Add JavaScript libraries
  3. Click on tab Repositories
  4. Select repository github.com/mementodatabase/script/data-sources
  5. Select discogs.js

Example of using this library:

var discogs = new Discogs("Consumer key" ,"Consumer secret" , "release");
var r = discogs.search(query); 
result(r, function(id) {return discogs.extra(id);});
  • Consumer key and consumer secret are the keys required to execute the request. They may be obtained at http://discogs.com/settings/developers.
  • After the script is written, it is necessary to map the object properties to the fields of the library. Available object properties are documented in the discogs API documentation. See an example of a response to a successful request at https://www.discogs.com/developers/#page:database,header:database-release.
  • If the search in the data source is done by barcode, use discogs.barcode(query) instead of discogs.search(query).