Chapter 8. Communication protocols

Table of Contents

8.1. Communicating over HTTP
8.1.1. Logging in
8.1.2. Logging out
8.1.3. Requesting a list of available repositories
8.1.4. Evaluating a SeRQL-select, RQL or RDQL query
8.1.5. Evaluating a SeRQL-construct query
8.1.6. Extracting RDF from a repository
8.1.7. Uploading data to a repository
8.1.8. Adding data from the web to a repository
8.1.9. Clearing a repository
8.1.10. Removing statements

In the previous chapter, we have seen how a Java application can communicate remotely with Sesame through the API. It is also possible to bypass this API (for example, when the client application is not a Java app) and communicate with a remote Sesame server directly through the HTTP protocol. In the first section of this chapter, the available actions and parameters for HTTP communication are explained.

8.1. Communicating over HTTP

In this section, the URL http://HOSTNAME/SESAME_DIR/ is used quite often. This URL is meant to indicate the location where Sesame is installed. HOSTNAME should be replaced by the name of the machine that is running Sesame. SESAME_DIR should be replaced by the directory where Sesame is installed (an empty string if Sesame is installed at the server's root directory).

To communicate with our public demo server, HOSTNAME should be replaced by www.openrdf.org and SESAME_DIR should be replaced by sesame. This yields http://www.openrdf.org/sesame/ as the location for the public demo server.

Sesame makes extensive use of multipart/form-data encoded HTTP-POST requests. The reason for this is that both HTTP-GET and x-www-form-url-encoded HTTP-POST requests can not properly handle non-ASCII characters. The %xx escape sequences used by this encoding is not able to represent byte values larger than 255.

All requests described below use multipart/form-data encoded HTTP-POST requests, unless specified otherwise. All string values are assumed to be encoded in UTF-8. Sesame switched to using this communication style in version 1.1-RC2. For now, the server will also be able to handle the previously used x-www-form-url-encoded requests to facilitate the transition of all third-party tools.

8.1.1. Logging in

8.1.1.1. Request

In order to get access to non-public repositories on a Sesame server, one needs to log in to it. To do this, a request should be sent to http://HOSTNAME/SESAME_DIR/servlets/login, supplying the following parameters:

Table 8.1. Parameters for login

ParameterRequired?Description
useryesThe user's login name.
passwordnoThe user's password. The password is only required in case the concerning user has a non-empty password.

8.1.1.2. Response

If the log in attempt succeeded, Sesame will respond with HTTP-code 200 ("OK") and will supply a cookie with a session ID that should be supplied for authentication in subsequent requests. The name of this cookie is sesame_sid.

8.1.1.3. Errors

If an invalid request is sent (e.g. the username is missing or the password was incorrect) Sesame will respond with an HTTP error code and a free format error message.

8.1.2. Logging out

8.1.2.1. Request

To log out from a Sesame server, one needs to send a request to http://HOSTNAME/SESAME_DIR/servlets/logout, supplying a session ID cookie that has been received earlier while logging in. Duplicate log out requests or log out requests for non-existing sessions are silently ignored.

8.1.2.2. Response

Sesame will respond with HTTP-code 200 ("OK") and a free format text telling you that you have been logged out.

8.1.3. Requesting a list of available repositories

8.1.3.1. Request

Information on available repositories can be acquired from Sesame by sending an HTTP-GET request to http://HOSTNAME/SESAME_DIR/servlets/listRepositories. This request requires no parameters, other than the optional session cookie.

8.1.3.2. Response

Sesame will respond with HTTP-code 200 ("OK"). The body of the response contains an XML-document describing the repositories available to the user that is currently logged in (if any). This XML-document will adhere to the following DTD:

<!ELEMENT repositorylist (repository)*>
<!ELEMENT repository     (title)>
<!ELEMENT title          (#PCDATA)>

<!ATTLIST repository
          id        ID           #REQUIRED
          readable  (true|false) #REQUIRED
          writeable (true|false) #REQUIRED>

The following is an example output with two repositories, one read-write and one read-only.

<?xml version="1.0" encoding="UTF-8"?>
<repositorylist>
    <repository id="repository1" readable="true" writeable="true">
        <title>Public repository</title>
    </repository>
    <repository id="repository2" readable="true" writeable="false">
        <title>Repository 2</title>
    </repository>
</repositorylist>

The value of the id attribute is the identifier of the repository. This identifier can be used in consecutive requests to Sesame to specify which repository that request is about.

8.1.4. Evaluating a SeRQL-select, RQL or RDQL query

8.1.4.1. Request

SeRQL-select, RQL and RDQL queries are all queries that return, in some way or another, sets of variable bindings or table-like results. SeRQL-construct queries are a different type of queries that return free-form RDF documents. Details about SeRQL-construct queries can be found in Evaluating a SeRQL-construct query.

SeRQL-select, RQL and RDQL queries can be sent to http://HOSTNAME/SESAME_DIR/servlets/evaluateTableQuery. The multipart/form-data encoded HTTP-POST method is the preferred method. HTTP-GET is also supported but can only be used if all parameter values consists of ASCII characters only. All requests can or must include the following parameters:

Table 8.2.  Parameters for evaluateTableQuery

ParameterRequired?Description
repositoryyesThe id of the repository to evaluate the query on.
queryyesThe query to evaluate.
queryLanguageyesThe query's language. Legal values are SeRQL, RQL and RDQL.
resultFormatnoDetermines the format of the result. Legal values are xml (the default value) for an XML-representation of the query result, html for a human-readable HTML-table presentation and rdf for an RDF-representation of the variable bindings.
serializationnoThe serialization to use when rdf is choosen as the resultFormat. Legal values are rdfxml (the default value), ntriples and turtle.

8.1.4.2. Response

If all went well, Sesame responds with HTTP-code 200 ("OK") and the results of the query evaluation. The resultFormat and serialization parameters determine the format of the results and the Content-Type header of the response:

Table 8.3. Response formats for SeRQL-select, RQL and RDQL queries

resultFormatserializationContent-typeResultComments
xml-text/xmlAn XML representation of the results table.This is the default format and is the best format for processing by other applications.
html-text/htmlAn HTML documentThe results are formatted in a human-readable (when displayed in a browser) HTML-table. This format should only be used by the Sesame web interface.
rdfrdfxmlapplication/rdf+xmlAn XML-encoded RDF documentThe result is an RDF document containing sets of variable bindings. This is an experimental format from Andy Seaborne and may be modified in the future. See http://www.w3.org/2003/03/rdfqr-tests/recording-query-results.html for details.
rdfntriplestext/plainAn N-Triples-encoded RDF documentThe result is identical to the previous one, but uses another encoding: N-Triples. See http://www.w3.org/TR/rdf-testcases/#ntriples for more information about N-Triples.
rdfturtleapplication/x-turtleA Turtle-encoded RDF documentThe result is identical to the previous two, but uses yet another encoding: Turtle (Terse RDF Triple Language). See http://www.ilrt.bris.ac.uk/discovery/2004/01/turtle/ for more information about Turtle.

If the selected result format is xml, the output will look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<tableQueryResult>
  <header>
    <columnName>X</columnName>
    <columnName>Y</columnName>
  </header>
  <tuple>
    <uri>http://www.foo.com/schema.rdf#change</uri>
    <literal>Change</literal>
  </tuple>
  <tuple>
    <bNode>node001</bNode>
    <literal datatype="http://www.w3.org/2001/XMLSchema#int">256</literal>
  </tuple>
</tableQueryResult>

The document starts with a header listing the names of all result columns. The names of these columns often represent variable names, but this is not guaranteed. Some query languages also allow you to use constants or functions in a query's projection. In that case, the name of the column will be the constant and the function, respectively.

Following the header, zero or more tuple-elements will follow, one for each query result (or row in the table). Each result consists of a sequence of URIs, bNodes (blank nodes, also know as anonymous nodes), and/or literals. The values appear in the order in which they were specified in the query, which is the same as the order in the header.

The example output above shows how URIs, bNodes and literals (with their optional language encoding or datatype) are exported. Next to these three, there is a fourth type of value that is only exported by the RQL engine: intersections of classes. If a query asks for a domain or range of a property, then this domain or range can consist of the intersection of zero or more classes. These intersections are exported in intersection elements containing zero or more URIs and bNodes:

<?xml version="1.0" encoding="UTF-8"?>
<tableQueryResult>
  <tuple>
    <uri>http://www.icom.com/schema1.rdf#creates</uri>
    <intersection>
      <uri>http://www.icom.com/schema1.rdf#Artifact</uri>
      <uri>http://www.schemas.org/general#Stuff</uri>
    </intersection>
  </tuple>
</tableQueryResult>

When a property doesn't have any range restrictions, its range will be an empty intersection element. Domains or ranges that consist of exactly one class are exported as normal resources (see first example), and not as intersections.

8.1.4.3. Errors

If an error occurred while the request was being processed, Sesame will send back an HTTP error code (i.e. not 200) and a free format error message.

8.1.5. Evaluating a SeRQL-construct query

8.1.5.1. Request

SeRQL-construct queries can be sent to http://HOSTNAME/SESAME_DIR/servlets/evaluateGraphQuery. The multipart/form-data encoded HTTP-POST method is the preferred method. HTTP-GET is also supported but can only be used if all parameter values consists of ASCII characters only. All requests can or must include the following parameters:

Table 8.4. Parameters for evaluateGraphQuery

ParameterRequired?Description
repositoryyesThe id of the repository to evaluate the query on.
queryyesThe query to evaluate.
queryLanguageyesThe query's language. Currently, the only legal value is SeRQL.
serializationnoDetermines the serialization format of the returned RDF document. Legal values are rdfxml (the default value), ntriples and turtle.

8.1.5.2. Response

If all went well, Sesame responds with HTTP-code 200 (i.e. "OK") and the results of the query evaluation. The serialization parameter determines the encoding of the returned RDF document and the Content-Type header of the response:

Table 8.5. RDF encodings for SeRQL-construct queries

SerializationContent-typeRDF encodingComments
rdfxmlapplication/rdf+xmlXML-encoded RDF.This is the official standard format for exchanging RDF documents. See http://www.w3.org/RDF/ for more info.
ntriplestext/plainN-TriplesThis is a quite verbose, line-based encoding that is the easiest one to parse. See http://www.w3.org/TR/rdf-testcases/#ntriples for more info.
turtleapplication/x-turtleTurtle Turtle stands for Terse RDF Triple Language and is developed by Dave Beckett to combine useful features of N3 and N-Triples into an easy-to-read-and-parse serialization format. See http://www.ilrt.bris.ac.uk/discovery/2004/01/turtle/ for more info.

What follows are three examples of one and the same RDF document (a possible query result) in different encodings.

XML-encoded RDF:

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF xml:lang="en" 
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:cult="http://www.icom.com/schema.rdf#">

  <cult:Cubist rdf:about="http://www.european-history.com/picasso.html">
    <cult:paints>
      <cult:Painting rdf:about="http://www.european-history.com/jpg/guernica03.jpg">
        <cult:technique>oil on canvas</cult:technique>
      </cult:Painting>
    </cult:paints>
    <cult:first_name>Pablo</cult:first_name>
    <cult:last_name>Picasso</cult:last_name>
  </cult:Cubist>
</rdf:RDF>

N-Triples:

<http://www.european-history.com/picasso.html> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.icom.com/schema.rdf#Cubist>.
<http://www.european-history.com/picasso.html> <http://www.icom.com/schema.rdf#paints> <http://www.european-history.com/jpg/guernica03.jpg>.
<http://www.european-history.com/jpg/guernica03.jpg> <http://www.w3.org/1999/02/22-rdf-syntax#type> <http://www.icom.com/schema.rdf#Painting>.
<http://www.european-history.com/jpg/guernica03.jpg> <http://www.icom.com/schema.rdf#technique> "oil on canvas"@en.
<http://www.european-history.com/picasso.html> <http://www.icom.com/schema.rdf#first_name> "Pablo"@en.
<http://www.european-history.com/picasso.html> <http://www.icom.com/schema.rdf#last_name> "Picasso"@en.

Turtle:

@prefix cult: <http://www.icom.com/schema.rdf#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

<http://www.european-history.com/jpg/guernica03.jpg> a cult:Painting ;
    cult:technique "oil on canvas"@en .
<http://www.european-history.com/picasso.html> a cult:Cubist ;
    cult:paints <http://www.european-history.com/jpg/guernica03.jpg> ;
    cult:first_name "Pablo"@en ;
    cult:last_name "Picasso"@en .

8.1.5.3. Errors

If an error occurred while the request was being processed, Sesame will send back an HTTP error code (i.e. not 200) and a free format error message.

8.1.6. Extracting RDF from a repository

8.1.6.1. Request

Ontologies and data can be extracted from Sesame by sending a request to http://HOSTNAME/SESAME_DIR/servlets/extractRDF. This operation supports both multipart/form-data encoded HTTP-POST requests as well as HTTP-GET requests. Note that the latter can not be used when one of the parameter values contains non-ASCII characters.

The request must or can have the following parameters:

Table 8.6. Parameters for extractRDF

ParameterRequired?Description
repositoryyesThe id of the repository to extract the data from.
schemanoRdfExport will export the schematical data if the value of this parameter is on. The schematical data will not be exported if this parameter is missing or if it has any other value.
datanoRdfExport will export the non-ontological data if the value of this parameter is on. This data will not be exported if this parameter is missing or if it has any other value.
explicitOnlynoRdfExport will only export the explicitly added statements if the value of this parameter is on. If this parameter is missing or if it has any other value, all relevant statements (both explicit and inferred) will be exported.
niceOutputnoRdfExport will invest extra effort into making the exported data more readable to humans (e.g. by sorting statements by their subject) if the value of this parameter is on. If this parameter is missing or if it has any other value, no such effort is made, allowing RdfExport to export the data as efficiently as possible.
serializationnoDetermines the serialization of the resulting RDF document. Legal values are rdfxml (the default value), ntriples and turtle.

So, to extract the ontological data only (both explicit and inferred) from a repository test-db on host www.openrdf.org you can send an HTTP-GET request to the url http://www.openrdf.org/sesame/servlets/extractRDF?repository=test-db&serialization=rdfxml&onto=on.

8.1.6.2. Response

If all went well, Sesame responds with HTTP-code 200 ("OK"). The body of the response contains the requested data. The serialization parameter determines the encoding and the Content-Type header of the response identically to what is described in Table 8.5, “RDF encodings for SeRQL-construct queries”.

8.1.6.3. Errors

If an invalid request is sent (e.g. the repository does not exist) Sesame will respond with an HTTP error code and a free format error message.

8.1.7. Uploading data to a repository

8.1.7.1. Request

Data can be uploaded to Sesame by sending it to http://HOSTNAME/SESAME_DIR/servlets/uploadData, supplying the following parameters:

Table 8.7. Parameters for uploadData

ParameterRequired?Description
repositoryyesThe id of the repository to which to add the data.
datayesThe RDF data that should be added to the repository. The format of the data should match the value of the dataFormat parameter.
dataFormatnoThe format of the data that is uploaded to Sesame. Legal values are rdfxml (the default value), ntriples and turtle.
baseURInoSpecifies the base URI of the data for resolving any relative URIs. This parameter defaults to foo:bar, which is fine if the uploaded data does not contain any relative URIs.
verifyDatanoThe uploaded data will be verified before it is actually added to a repository if the value of this parameter is on. If this parameter is missing or if it has any other value, the data will be added directly to the repository. Verification of the data should only be disabled if the data to upload is guaranteed to be correct.
resultFormatnoDetermines the response format. Legal values are xml (the default value) and html.

8.1.7.2. Response

The response is always an HTTP-code 200 ("OK"). Streaming feedback is given on the progress, and warnings and errors are given if the data contains errors. If the selected format was html, then this feedback will be formatted in human-readable HTML (when displayed in a browser). If the selected format was xml, then the feedback will adhere to the following DTD:

<!ELEMENT transaction   (status | notification | warning | error)*>
<!ELEMENT status        (msg, line?, column?)>
<!ELEMENT notification  (msg, line?, column?, statement?)>
<!ELEMENT warning       (msg, line?, column?, statement?)>
<!ELEMENT error         (msg, line?, column?, statement?)>

<!ELEMENT msg           (#PCDATA)>
<!ELEMENT line          (#PCDATA)>
<!ELEMENT column        (#PCDATA)>

<!ELEMENT statement     ( (uri|bNode), uri, (uri|bNode|literal) )>
<!ELEMENT uri           (#PCDATA)>
<!ELEMENT bNode         (#PCDATA)>
<!ELEMENT literal       (#PCDATA)>
<!ATTLIST literal
      xml:lang CDATA #IMPLIED
      datatype CDATA #IMPLIED>

The following is a simple example of what the feedback might look like when incorrect data is uploaded to Sesame:

<?xml version="1.0" encoding="UTF-8"?>
<transaction>
    <status>
        <msg>Loading data</msg>
    </status>
    <status>
        <msg>Data loaded (756 bytes)</msg>
    </status>
    <status>
        <msg>Checking data for errors</msg>
    </status>
    <error>
      <msg>Not a legal integer: Hello World!</msg>
        <line>483</line>
        <column>42</column>
        <statement>
            <bNode>node1234</bNode>
            <uri>http://example.org/schema.rdf#age</uri>
            <literal datatype="http://www.w3.org/2001/XMLSchema#int">Hello World!</literal>
        </statement>
    </error>
</transaction>

8.1.7.3. Errors

Errors in the requests are signaled using HTTP error codes. Errors in the data are expressed in the streaming feedback.

8.1.8. Adding data from the web to a repository

8.1.8.1. Request

Data can also be added to a repository by specifying a URL that points to the RDF data that should be added. To do this, send an HTTP-POST request to http://HOSTNAME/SESAME_DIR/servlets/uploadURL, supplying the following parameters:

Table 8.8. Parameters for uploadData

ParameterRequired?Description
repositoryyesThe id of the repository to which to add the data.
urlyesA URL pointing at the RDF data that should be added to the repository. The format of the data at the specified location should match the value of the dataFormat parameter.
dataFormatnoThe format of the data that is at the specified URL. Legal values are rdfxml (the default value), ntriples and turtle.
baseURInoSpecifies the base URI of the data for resolving any relative URIs. This parameter defaults to the value of url.
verifyDatanoThe data at the specified URL will be verified before it is actually added to a repository if the value of this parameter is on. If this parameter is missing or if it has any other value, the data will be added directly to the repository. Verification of the data should only be disabled if the data to upload is guaranteed to be correct.
resultFormatnoDetermines the response format. Legal values are xml (the default value) and html.

8.1.8.2. Response

The response format is identical to what is described in Data upload response format.

8.1.8.3. Errors

Errors in the requests are signaled using HTTP error codes. Errors in the data are expressed in the streaming feedback.

8.1.9. Clearing a repository

8.1.9.1. Request

Data can be removed from Sesame by sending a request to http://HOSTNAME/SESAME_DIR/servlets/clearRepository. The following parameters are relevant to clearing a repository:

Table 8.9. Parameters for clearRepository

ParameterRequired?Description
repositoryyesThe id of the repository that should be cleared.
resultFormatnoDetermines the response format. Legal values are xml (the default value) and html.

8.1.9.2. Response

Identical to the reponse when adding data to a repository. See See Data upload response format. for more information.

8.1.9.3. Errors

Errors in the requests are signaled using HTTP error codes. Errors that occured while clearing the repository are expressed in the streaming feedback.

8.1.10. Removing statements

8.1.10.1. Request

Data can be removed from Sesame by sending a requests to http://HOSTNAME/SESAME_DIR/servlets/removeStatements. Currently, Sesame only support removing of statements using statement patterns: one can specify a subject, predicate and/or object and all statements that match that pattern will be removed. The following parameters are relevant to the removal of statements:

Table 8.10. Parameters for removeStatements

ParameterRequired?Description
repositoryyesThe id of the repository from which the statements should be removed.
subjectnoAn N-Triples encoded URI or bNode specifying the subject of the statements that should be removed. If not specified, statements with any subject will be removed.
predicatenoAn N-Triples encoded URI specifying the predicate of the statements that should be removed. If not specified, statements with any predicate will be removed.
objectnoAn N-Triples encoded URI, bNode or literal specifying the object of the statements that should be removed. If not specified, statements with any object will be removed.
resultFormatnoDetermines the result format. Legal values are xml (the default value) and html.

8.1.10.2. Response

Identical to the reponse when adding data to a repository. See Data upload response format for more information.

8.1.10.3. Errors

Errors in the requests are signaled using HTTP error codes. Errors that occured while removing statements from the repository are expressed in the streaming feedback.