Web Services
Core CMD model for Web Service descriptions
Introduction

Within the CLARIN infrastructure a growing number of web services is becoming available. To answer linguistic research questions advanced workflows of these services will be created. Various national CLARIN initiatives are using or building workflow engines to specify and execute these workflows. In general these workflow engines execute web services based on metadata stored in an engine specific repository. This core CMD model for CLARIN web service descriptions is based on the overlap between the metadata schemes of these repositories.

UML model


( View the UML model as
SVG, EMF, JPEG, PNG or PDF )

CMD profile

Based on this UML model a CMD profile has been created. However, notice as CMD doesn't support object oriented features like abstract classes and inheritance this profile isn't a one-to-one transformation.

CMD profile/component/elementtypecardinality (min:max)documentation
CLARINWebServiceCLARINWebService (CLARIN core WS descr - version 1.0.2)1:1
ServiceService (CLARIN core WS descr - version 1.0.2)1:unbounded
Namestring1:1The name of the web service or set of web services.
Descriptionstring0:1A description of the web service or set of web services.
ServiceDescriptionLocationcomponent1:1
LocationanyURI0:1
Operationscomponent1:1
OperationOperation (CLARIN core WS descr - version 1.0.2)1:unbounded
Namestring1:1This Operation Name should be resolvable in the Service ServiceDescription, i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1The Operation Description should describe what this operation does to the input to produce the output.
Inputcomponent0:1
ParameterGroupParameterGroup (CLARIN core WS descr - version 1.0.2)0:unbounded
Namestring1:1This ParameterGroup Name should in general be resolvable in the Service ServiceDescription, i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1Describes the role of this ParameterGroup for this web service operation.
MIMETypestring0:1The MIME type of this ParameterGroup, which indicates that its one input resource containing several input Parameters.
DataTypestring0:1The data type of this ParameterGroup.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this ParameterGroup.
SemanticTypestring0:1A type describing the semantics of this parameter in its context. For example, an input parameter might be an WS specific project name, given to a project created by an earlier WS call. In that case the name of the parameter might be 'project', its data type might be 'xsd:string', its data category /projectName/ and its semantic type 'myWSApp.project'.
Parameterscomponent1:1
ParameterParameter (CLARIN core WS descr - version 1.0.2)1:unbounded
Namestring1:1This Parameter Name, unless the Parameter is part of a ParameterGroup, should be resolvable in the Service ServiceDescription, , i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1Describes the role of this Parameter for this web service operation.
MIMETypestring0:1The MIME type of this Parameter, which indicates that its an input resource on its own.
DataTypestring0:1The data type of this Parameter. In general this should be an XSD data type, e.g., xsd:string or xsd:anyURI.
isConfigurationParameterboolean0:1Set this flag to true if the parameter configures the operation of the web services, which is something else then the input the operation works upon. For example, the input of an operation can be a series of sensor values while a configuration parameter might be a treshhold or a weight used in the calculation done on this sensor values.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter.
SemanticTypestring0:1A type describing the semantics of this parameter in its context. For example, an input parameter might be an WS specific project name, given to a project created by an earlier WS call. In that case the name of the parameter might be 'project', its data type might be 'xsd:string', its data category /projectName/ and its semantic type 'myWSApp.project'.
Valuescomponent0:1
ParameterValuecomponent1:unbounded
Valuestring1:1A possible literal value of a parameter.
Descriptionstring0:1A description of the meaning of a possible literal value of a parameter.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter Value.
ParameterParameter (CLARIN core WS descr - version 1.0.2)0:unbounded
Namestring1:1This Parameter Name, unless the Parameter is part of a ParameterGroup, should be resolvable in the Service ServiceDescription, , i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1Describes the role of this Parameter for this web service operation.
MIMETypestring0:1The MIME type of this Parameter, which indicates that its an input resource on its own.
DataTypestring0:1The data type of this Parameter. In general this should be an XSD data type, e.g., xsd:string or xsd:anyURI.
isConfigurationParameterboolean0:1Set this flag to true if the parameter configures the operation of the web services, which is something else then the input the operation works upon. For example, the input of an operation can be a series of sensor values while a configuration parameter might be a treshhold or a weight used in the calculation done on this sensor values.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter.
SemanticTypestring0:1A type describing the semantics of this parameter in its context. For example, an input parameter might be an WS specific project name, given to a project created by an earlier WS call. In that case the name of the parameter might be 'project', its data type might be 'xsd:string', its data category /projectName/ and its semantic type 'myWSApp.project'.
Valuescomponent0:1
ParameterValuecomponent1:unbounded
Valuestring1:1A possible literal value of a parameter.
Descriptionstring0:1A description of the meaning of a possible literal value of a parameter.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter Value.
Outputcomponent1:1
ParameterGroupParameterGroup (CLARIN core WS descr - version 1.0.2)0:unbounded
Namestring1:1This ParameterGroup Name should in general be resolvable in the Service ServiceDescription, i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1Describes the role of this ParameterGroup for this web service operation.
MIMETypestring0:1The MIME type of this ParameterGroup, which indicates that its one input resource containing several input Parameters.
DataTypestring0:1The data type of this ParameterGroup.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this ParameterGroup.
SemanticTypestring0:1A type describing the semantics of this parameter in its context. For example, an input parameter might be an WS specific project name, given to a project created by an earlier WS call. In that case the name of the parameter might be 'project', its data type might be 'xsd:string', its data category /projectName/ and its semantic type 'myWSApp.project'.
Parameterscomponent1:1
ParameterParameter (CLARIN core WS descr - version 1.0.2)1:unbounded
Namestring1:1This Parameter Name, unless the Parameter is part of a ParameterGroup, should be resolvable in the Service ServiceDescription, , i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1Describes the role of this Parameter for this web service operation.
MIMETypestring0:1The MIME type of this Parameter, which indicates that its an input resource on its own.
DataTypestring0:1The data type of this Parameter. In general this should be an XSD data type, e.g., xsd:string or xsd:anyURI.
isConfigurationParameterboolean0:1Set this flag to true if the parameter configures the operation of the web services, which is something else then the input the operation works upon. For example, the input of an operation can be a series of sensor values while a configuration parameter might be a treshhold or a weight used in the calculation done on this sensor values.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter.
SemanticTypestring0:1A type describing the semantics of this parameter in its context. For example, an input parameter might be an WS specific project name, given to a project created by an earlier WS call. In that case the name of the parameter might be 'project', its data type might be 'xsd:string', its data category /projectName/ and its semantic type 'myWSApp.project'.
Valuescomponent0:1
ParameterValuecomponent1:unbounded
Valuestring1:1A possible literal value of a parameter.
Descriptionstring0:1A description of the meaning of a possible literal value of a parameter.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter Value.
ParameterParameter (CLARIN core WS descr - version 1.0.2)0:unbounded
Namestring1:1This Parameter Name, unless the Parameter is part of a ParameterGroup, should be resolvable in the Service ServiceDescription, , i.e., the WSDL or WADL document describing how to technically invoke the service.
Descriptionstring0:1Describes the role of this Parameter for this web service operation.
MIMETypestring0:1The MIME type of this Parameter, which indicates that its an input resource on its own.
DataTypestring0:1The data type of this Parameter. In general this should be an XSD data type, e.g., xsd:string or xsd:anyURI.
isConfigurationParameterboolean0:1Set this flag to true if the parameter configures the operation of the web services, which is something else then the input the operation works upon. For example, the input of an operation can be a series of sensor values while a configuration parameter might be a treshhold or a weight used in the calculation done on this sensor values.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter.
SemanticTypestring0:1A type describing the semantics of this parameter in its context. For example, an input parameter might be an WS specific project name, given to a project created by an earlier WS call. In that case the name of the parameter might be 'project', its data type might be 'xsd:string', its data category /projectName/ and its semantic type 'myWSApp.project'.
Valuescomponent0:1
ParameterValuecomponent1:unbounded
Valuestring1:1A possible literal value of a parameter.
Descriptionstring0:1A description of the meaning of a possible literal value of a parameter.
DataCategoryanyURI0:1The PID refering to the (ISOcat) data category specifying this Parameter Value.

( View the CMD profile in the registry or download the profile as XML or XSD )

Notes:

  1. In an instance of the UML model Parameter and ParameterGroup instances would me able to mix in the Input, Output and Parameters lists, but in the CMD model ParameterGroup instances have to precede Parameter instances
  2. In the CMD model ServiceDescriptionLocation is a component instead of an element to enable the use of a CMD ResourceProxy to point to the WSDL or WADL file
    Example use of CMD ResourceProxy
    <CMD xmlns="http://www.clarin.eu/cmd/" ... CMDVersion="1.1">
        
    <Header />
        
    <Resources>
            
    <ResourceProxyList>
                
    <ResourceProxy id="r1">
                    
    <!--the WSDL MIME type is application/wsdl+xml-->
                    
    <ResourceType mimetype="application/vnd.sun.wadl+xml">Resource</ResourceType>
                    
    <ResourceRef>uri:myServiceDescription</ResourceRef>
                
    </ResourceProxy>
            
    </ResourceProxyList>
            
    <JournalFileProxyList />
            
    <ResourceRelationList />
        
    </Resources>
        
    <Components>
            
    <myProfile>
                
    <Service CoreVersion="1.0">
                    
    <Name>myService</Name>
                    
    <ServiceDescriptionLocation ref="r1" />
                    
    <Operations>
                        ...
                    
    </Operations>
                
    </Service>
            
    </myProfile>
        
    </Components>
    </CMD>
    However, since version 1.0.2 also an, optional, Location element is provided, which can be used instead of the ResourceProxy. Please use only one of these options to prevent inconsistencies.
Usage

A web service repository shouldn't directly offer its services in instances of the core model. Instead a repository specific profile should be created. This repository specific profile can extend the core model, e.g., it can add additional CMD elements and components. This can be done by editing copies of the components of the core model. The current versions of the core components are in the CLARIN core WS descr - version 1.0.2 group in the public space in the ComponentRegistry.

The following public CMD profiles are valid extensions to the core model and are or will be used by CLARIN web service registries, workflow and/or chaining engines:

ToolService - CLARIN-NL Tool/Service descr - version 1.0
CMD profile in use within CLARIN-NL/VL and to be used within the TTNWW project.
ToolService - CLARIN-NL Tool/Service descr - DANS - version 1.0
DANS version of the CLARIN-NL/VL ToolService profile, which allows an optional instance of the DANS-DC-metadata component.
WebLichtWebService - NaLiDa
From version 2.0 on CLARIN-D's WebLicht supports the CMD core model for CLARIN Web Services. A dedicated online editor has been created and is available to CLARIN-D developers (look in the archive or ask on the CLARIN-D developers mailinglist if you don't know the credentials).

Notes:

  1. Notice that the profile name can be different from the core profile, e.g., ToolService, as the associated element name will be skipped during validation.
Validation

When an instance of the repository specific profile is stripped of all the extensions it should still be a valid instance of the core model. This validation can be done by uploading the instance using the form below.

Validate a CMD instance

Example

This example shows how the CLARIN-NL repository specific ToolService profile has been created. This profile mainly adds the TechnicalMetadata component to the Parameter component of the core model.

  1. Add the TechnicalMetadata component to the core Parameter component:
    1. Open the core Parameter component in the editor
    2. Change the group name of the component, so you can find the edited component back
      • Do not change the component name, as that will change the element names and make the instances invalid!
    3. Add the TechnicalMetadata component
    4. Save the edited Parameter component in your own workspace (see
  2. Recusively update each component that refers to an updated component:
    1. Update the ParameterGroup component:
      1. Open the core ParameterGroup component in the editor
      2. Change the group name of the component, so you can find the edited component back
        • Do not change the component name, as that will change the element names and make the instanced invalid!
      3. Replace the Parameter component by the edited Parameter component from your own workspace
        • Recreate the cardinality constraints so they stay valid in the core model, i.e., you can have a lower maximum cardinality but not a lower minimum cardinality!
      4. Save the edited ParameterGroup component in your own workspace
    2. Update the Operation component:
      1. Open the core Operation component in the editor
      2. Change the group name of the component, so you can find the edited component back
        • Do not change the component name, as that will change the element names and make the instanced invalid!
      3. Replace the Parameter and ParameterGroup components by the edited Parameter and ParameterGroup components from your own workspace
        • Recreate the cardinality constraints so they stay valid in the core model, i.e., you can have a lower maximum cardinality but not a lower minimum cardinality!
      4. Save the edited Operation component in your own workspace
    3. Update the Service component:
      1. Open the core Service component in the editor
      2. Change the group name of the component, so you can find the edited component back
        • Do not change the component name, as that will change the element names and make the instanced invalid!
      3. Replace the Operation component by the edited Operation components from your own workspace
        • Recreate the cardinality constraints so they stay valid in the core model, i.e., you can have a lower maximum cardinality but not a lower minimum cardinality!
      4. Save the edited Service component in your own workspace
  3. Create your own profile which uses the Service component from your own workspace
  4. Check if an instance of your profile is also a valid instance of the core model:
    1. Create a valid instance based on the schema of your own profile
    2. Validate the instance also against the core model using the form above

The results of the CLARIN-NL Adelheid project has been described using this ToolService profile and is thus also a valid instantiation of the core model:

  1. CMDI metadata file: http://catalog.clarin.eu/metadata/cmdi/services/Adelheid.cmdi
  2. referenced WADL files:
    1. adelheid/main: http://catalog.clarin.eu/adelheidws/wadl/main.wadl
    2. adelheid/lexicon: http://catalog.clarin.eu/adelheidws/wadl/lexicon.wadl
    3. adelheid/tokenizer: http://catalog.clarin.eu/adelheidws/wadl/tokenizer.wadl

Notes:

  1. The links in the example above are based on version 1.0.1 of the core model and version 1.0.0 of the ToolService profile.
Frequently Asked Questions

Why do we need a technical service description, e.g., a WADL or WSDL file, next to the CMD metadata to fully document a web service?

The CMD metadata for web services focusses on the semantic description of a service, i.e., what goes in and what comes out, but it doesn't specifiy how to technically invoke the web service. Should you, for example, pass on the parameters into the query string and use the GET HTTP verb to invoke the service and receive the result? Or should one do a multi-part POST? Or should one wrap the input in a SOAP envelope and POST it to the web service? One could extend the core model to make this explicit, but there are already many of these proposals, e.g., WSDL and WADL, which can be reused. To invoke aribtrary web services a workflow or chaining engine needs to know this. For a more in-depth discussion see the Metadata2012 paper on the core model.

The core model doesn't prescribe a technical service description language or Interface Definition Language (IDL), why not?

For SOAP web services there is a standard, i.e., WSDL (2), but for REST-based web services there isn't. Sun submitted WADL some years ago, but it never made it to a recommendation. So currently there are several competing proposals:

Publications

  1. M. Windhouwer, D. Broeder, D. van Uytvanck. A CMD Core Model for CLARIN Web Services. In the Proceedings of the Metadata2012 Workshop on Describing Language Resources with Metadata: Towards Flexibility and Interoperability in the Documentation of Language Resources. At LREC 2012, Istanbul, Turkey, May 22, 2012. (SlideShare)

Contact

For any additional information feel free to contact:

 Menzo Windhouwer (CLARIN ERIC/Meertens Institute)

Acknowledgements

This core CMD model is based on information provided by and feedback from:

  • Marc Kemps-Snijders (Meertens Institute)
  • Marta Villegas (Universitat Pompeu Fabra)
  • Thomas Zastrow (University of Tübingen)
TODO's
  1. Validation is still based on CMDI 1.1, but should move to 1.2.
  2. Concepts have been created, but often don't have a definition.
Change log

[2017-10-12] this page

  1. The validator can handle CMDI 1.2 records, but validation is still based on 1.1.
  2. Updated to refer to and describe the use of version 1.0.2 of the profile

[2017-09-28] CLARIN core WS descr - version 1.0.2

  1. Added Location element in ServiceDescriptionLocation to be compliant with CMDI 1.2

[2017-08-17] this page

  1. Updated various links
  2. Prepare move from MPI to CLARIN

[2011-12-19] CLARIN core WS descr - version 1.0.1

  1. Inplace update of the component to allow multilingual descriptions

[2011-11-16] CLARIN core WS descr - version 1.0

  1. The CoreVersion attribute on the Service component now indicates the version (currently 1.0) of the core model that is instantiated
  2. Location attribute has been renamed to ServiceDescriptionLocation to indicate that it should point to a WSDL or WADL description of the service(s)
  3. ServiceDescriptionLocation is realised as a CMD component to enable the use of a CMD resource proxy for the URI of the WSDL or WADL document
  4. Recipe attribute has been removed as the ServiceDescriptionLocation now points to a machine readable recipe, i.e., a WSDL or WADL file, to invoke a service
  5. Name attribute of the AbstractParameter class should point to a parameter described in the WSDL or WADL file if the Parameter(Group) has a direct correspondent
  6. Name attribute of the Operation class should point to a service described in the WSDL or WADL file
  7. SemanticType attribute has been added to the AsbtractParameter class to fine tune output/input parameter matching
  8. DataCategory attribute has moved to the AbstractParameter class to allow references to container data categories from ParameterGroup

[2011-08-08] CLARIN core WS descr (tentative)

  1. Tentative release of the profile and components