This blog is (once more) about Integration Gateway in SAP Mobile Platform 3.0.
It is meant for those of you who have already created OData services based on Integration Gateway in SAP Mobile Platform 3.0 (SMP).
If you want your OData service to support filtering, you have to implement it in the custom script.
This tutorial explains how it is done using the ExpressionVisitor interface, which is provided by Olingo.
By the way, this tutorial is based on SMP SP05.
We’re not going to start from scratch. So in order to follow this tutorial, you need to have the project in place that is created in my previous tutorial.
Prerequisites
Preparation
The following steps were done in the previous tutorial:
- Create implementation project
- Define OData model
- Generate custom code for Groovy
- implement the processResponseData() method
- Deploy it to SMP
- Create/configure the destination
- Test the query operation.
Filter Implementation
Overview:
- Retrieve the Objects UriInfo and FilterExpression.
- Create Visitor class that implements the ExpressionVisitor interface
- Implement the 4 required methods
- Invoke the visitor
- Use the result to manipulate the relative URI
Step: Callback method processRequestData()
Implementing the callback method processRequestData() that is provided by the Integration Gateway Framework and which is generated into our custom script.
We have to understand:
This method is called, before the REST service (= the data source) is invoked.
The order is:
The user calls our OData service in SMP. E.g. https://<host>/gateway/odata/SAP/<SERVICE>/StreetSet?$filter=streetType eq ’PEDESTRIAN’
The OData call is received by Integration Gateway.
Integration Gateway requires our help, therefore it passes the OData-URI (which was called by the user) into our custom script, in the processRequestData() method
Here, we have to change the URI, because the filter statement in OData language is not understood by the REST service.
After changing the URI, we give it back to Integration Gateway.
Then Integration Gateway can safely call the REST service and will get the filtered data.
At this point we need to check the documentation of our REST service:
http://www.gisgraphy.com/documentation/user-guide.htm#streetwebservice
For this tutorial, I’ve chosen the streettype for filtering.
Example filter:
&streettype=PEDESTRIAN
The full URL:
http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41&streettype=PEDESTRIAN
Note:
Please ignore that this URL actually contains 3 filters.
The background is that when we've created the Eclipse project, we’ve configured following URL
http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41 as REST service-URL to provide data.
So, since the first 2 filters belong to the base URL (as we've defined it), we have to ignore these first 2 filters in our tutorial
So we have the base URL:
http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41
and we add one filter to it:
&streettype=PEDESTRIAN
Step: Object UriInfo
The Olingo API provides the interface UriInfo
We need it, because this object contains information about the URI, which includes system query options.
The part of the URI, in which we’re interested, is the filter statement.
Step: Object FilterExpression
The Olingo API provides the interface FilterExpression
We need it in order to parse the OData filter statement.
We obtain the filter statement by asking the UriInfo for the FilterExpression:
FilterExpression filterExpression = uriInfo.getFilter();
The filterExpression object contains the actual filter that is specified in the URL of the invoked OData service on SMP.
This is the $filter parameter, as specified in OData.
Example:
If the full URL of the OData service invocation is as follows:
https://<SMPserver>/gateway/odata/SAP/<YOUR_SERVICE>/StreetSet?$filter=streetType eq ’PEDESTRIAN’
then the filterExpression object contains the information about
filter=streetType eq ’PEDESTRIAN’
If the full URL of the OData service invocation is as follows (just as an example):
https://<server>/gateway/odata/SAP/<YOUR_SERVICE>/StreetSet?$format=json
then the filterExpression object is null
So we have to explicitly handle the case that no filter is desired:
if (filterExpression == null){
return;
}
The filterExpression object is required to invoke the ExpressionVisitor.
Step: Interface ExpressionVisitor
The Olingo API provides the interface ExpressionVisitor
This interface has to be implemented by us.
Here it is where the filter implementation is realized.
The ExpressionVisitor is required, because a filter statement can be complex.
The Olingo framework traverses all the filter statements in the OData service call and passes the segments to us in the different methods of the interface.
In our script, we create an inner class that implement the ExpressionVisitor interface:
defclass RestExpressionVisitor implements ExpressionVisitor{
We let Eclipse add the methods which have to be implemented.
Most of them can be ignored.
In following steps, we'll have a closer look at the relevant methods.
visitProperty()
visitLiteral()
visitBinary()
visitFilterExpression()
They are invoked in this order.
Example:
What happens, if our OData service is invoked with $filter=streetType eq ’PEDESTRIAN’
The methods are invoked in the following order:
| visitProperty() | Here we have to deal with the property, e.g. | streetType |
| visitLiteral() | Here we have to deal with the literal, e.g. | ‘PEDESTRIAN’ |
| visitBinary() | Here we have to deal with the operator, e.g. | eq |
| visitFilterExpression() | Here we have to deal with our filterExpression, e.g. | streetType eq ‘PEDESTRIAN’ |
Note:
These sample values on the right column depend on the implementation.
Second example:
What happens, if our OData service is invoked with $filter=streetType eq ’PEDESTRIAN’ and name eq ‘East Lake Street’
| visitProperty() | Deal with the property, e.g. | streetType |
| visitLiteral() | Deal with the literal, e.g. | ‘PEDESTRIAN’ |
| visitBinary() | Deal with the operator, e.g. | eq |
| visitProperty() | Deal with the second property, e.g. | name |
| visitLiteral() | Deal with the second literal, e.g. | ‘East Lake Street’ |
| visitBinary() | Deal with the operator, e.g. | eq |
| visitBinary() | Deal with the operator, e.g. | and |
| visitFilterExpression() | Deal with our filterExpression, e.g. | streetType eq ‘PEDESTRIAN’ and name eq ‘East Lake Street’ |
Note:
Again, these sample values depend on the implementation
We can see that the visitor traverses all segments and finally calls for the final expression
Note that In this tutorial we cover only simple filter statement.
Now let’s implement the methods for our example
Step: Method visitProperty()
The method signature:
Object visitProperty(PropertyExpression propertyExpression, String uriLiteral, EdmTyped edmTyped)
In this method, we get the possibility to analyze the property mentioned in the filter expression.
In our example:
$filter=streetType eq ’PEDESTRIAN’
Here, the text streetType is the name of a property (in the OData model).
When implementing the visitProperty() method, we can access the property as an object as follows:
EdmProperty edmProperty = (EdmProperty) edmTyped;
And the name of the property:
String propertyName = edmProperty.getName();
So what?
In our REST example service we really need this method.
Why?
Because the parameter, that is supported by the REST service is: streettype
(see docu)
but the name of the property (in both the OData model and the REST service) is: streetType
See it? They differ.
Without converting the OData property name to the REST parameter name, the filtering doesn't work.
But we can make use of the visitProperty() method in order to do a mapping from property name to parameter name
String restParameterName = null;
if(propertyName.equals("streetType")){ // we get the OData property name
restParameterName = "streettype"; // we convert it to REST parameter name
}else{
restParameterName = propertyName;
}
Our method has to return the name of the parameter that is used by the REST service.
See the sample implementation of the method here:
@Override
public Object visitProperty(PropertyExpression propertyExpression, String uriLiteral, EdmTyped edmTyped) {
if(! (edmTyped instanceof EdmProperty)){
thrownew RuntimeException("Unexpected type");
}
EdmProperty edmProperty = (EdmProperty) edmTyped;
String propertyName = null;
try {
propertyName = edmProperty.getName();
} catch (EdmException e) {
((ILogger)log).logErrors(LogMessage.TechnicalError, "Failed to read property");
thrownew RuntimeException("Unable to access name of property", e);
}
String restParameterName = null;
if(propertyName.equals("streetType")){ //in the REST service, the param is different
restParameterName = "streettype";
}else{
restParameterName = propertyName;
}
return restParameterName; // e.g. will lead to streettype=PEDESTRIAN
}
Step: Method visitLiteral()
The method signature:
Object visitLiteral(LiteralExpression literalExpression, EdmLiteral edmLiteral)
In this method, we get 2 objects, but we focus on the second only:
EdmLiteral can be used for obtaining the literal, which is the pure text of the filter.
What do we have to do?
As we’ve seen above, the syntax of our REST service for filtering is:
&<parametername>=<value>
Note that the value doesn’t require inverted comma.
![]()
-> here we have the difference to OData
So in our example the OData segment ’PEDESTRIAN’
Has to be converted to the REST segment PEDESTRIAN
(without inverted commas)
This is easy to implement: just remove the inverted commas.
And it is even easier to implement, because the following statement returns the value without inverted comma
edmLiteral.getLiteral()
So our implementation looks as follows:
@Override
public Object visitLiteral(LiteralExpression literalExpression, EdmLiteral edmLiteral) {
return edmLiteral.getLiteral();
}
Step: Method visitBinary()
The method signature:
Object visitBinary(BinaryExpression binaryExpression, BinaryOperator operator, Object leftSide, Object rightSide)
In this method, we have to convert the operator.
As we’ve seen above, the syntax of our REST service for filtering is:
&<parametername>=<value>
The operator is: =
![]()
-> this is the difference to OData
So in our example the OData operator: eq
Has to be converted to the REST operator: =
This has to be done for all supported operators, so we have to care about each operator in a switch statement.
Note that Olingo provides an enum for the OData operators:
org.apache.olingo.odata2.api.uri.expression.BinaryOperator
We get it as parameter in the method.
At the end, we have to return the whole expression.
In our tutorial, let’s support only 2 operators (it depends of course on the chosen REST service), to allow the above service invocation.
public Object visitBinary(BinaryExpression binaryExpression, BinaryOperator operator, Object leftSide, Object rightSide) {
String operatorForREST = "";
switch(operator){
case BinaryOperator.EQ:
operatorForREST = "=";
break;
case BinaryOperator.AND:
operatorForREST = "&";
break;
default:
thrownew RuntimeException("Not supported: " + operator.toUriLiteral());
}
return leftSide + operatorForREST + rightSide;
}
Step: Method visitFilterExpression()
The method signature:
Object visitFilterExpression(FilterExpression filterExpression, String expressionString, Object expression)
In this method, we have the chance to check the final filter expression.
The parameter expression contains our REST filter expression, as we've defined it in the previous visitor-methods.
E.g. streettype=PEDESTRIAN
What are we going to do here?
In a simple example, we would simply return the instance of expression.
Our REST sample service is a bit special, because our so-called base-URI already contains some parameters. As explained above, we use it simply to get some data. The fact that the REST service needs 2 parameters for geolocation, doesn’t disturb us.
It only forces us to do a little work in the visitFilterExpresission() method.
The reason is:
Our expression looks fine: streettype=PEDESTRIAN
But it has to be appended to the base URL, which already contains parameters:
http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41
So before appending our expression, we have to insert a &
That’s all.
Our sample implementation:
public Object visitFilterExpression(FilterExpression filterExpression, String expressionString, Object expression) {
if(expression instanceof String){
String expressionAsString = (String)expression;
if(! expressionAsString.startsWith("&")){
expressionAsString = "&" + expressionAsString;
return expressionAsString;
}
}
return expression;
}
Step: Use ExpressionVisitor
In the above steps, we've created the RestExpressionVisitor class, that implements the required interface and the necessary methods.
Now we can instantiate and invoke it:
RestExpressionVisitor visitor = new RestExpressionVisitor();
Object restFilterObject = null;
try {
restFilterObject = filterExpression.accept(visitor);
} catch (ExceptionVisitExpression e) {
log.logErrors(LogMessage.TechnicalError, "Failed calling accept()");
return;
} catch(ODataApplicationException oe){
log.logErrors(LogMessage.TechnicalError, "Failed calling accept()");
return;
}
The returned restFilterObject is the result of our effort: the REST-izised version of our OData-filter.
In our sample implementation, the resulting value is the following string:
&streettype=PEDESTRIAN&name=East Lake Street
What to do with it?
Remember that we’re implementing the callback method processRequestData(), where we’re supposed to manipulate the backend-URL, before it is called.
So we have to build the URI containing the filter string.
Usually, it is formed as follows:
<relativeURI> + ? + <filterString>
In our special case, the relativeURI already contains the ?
So we don’t need to add it.
We get the relativeURI string from the header, which is passed in the method signature in the message object.
In our example, the relative URI woud be:
/street/streetsearch?lat=41.89&lng=-87.64
The code looks as follows:
String relativeUri = getRelativeUri(message);
relativeUri = relativeUri + (String)restFilterObject;
and the code to obtain the relativeURI:
def String getRelativeUri(Message message){
Map headers = message.getHeaders();
Object object = headers.get("RelativeUri");
if(object instanceof String){
return (String)object;
}
returnnull;
}
The last step to do in the processRequestData() method is: to pass the new relativeUri to the Integration Gateway framework.
This is done by setting the header:
defvoid setRelativeUri(Message message, String newUri){
message.setHeader("RelativeUri", newUri);
}
Final: Run
That’s it.
At least, that’s it for the filter implementation in the method processRequestData().
Which is the focus of this blog.
We haven’t talked about the processResponseData(), as this was the focus of my previous blog
Note that you also need to add the required import statements and also to add required bundles.
I’m pasting the full script below, and I'm also attaching the relevant files of my Eclipse project.
Note that the attached files are based on SP05. So if you use them for a different version, you have to adapt them, e.g. update the dependencies in the manifest file.
Please refer to my previous blogs for detailed information (see Links section).
Testing the filtering
After generate and deploy the designtime artifact and after configuring the service on SMP, you can try calling several URLs with different filters:
First test without filter:
https://<host>:8083/gateway/odata/SAP/<YOUR_SERVICE>;v=1/StreetSet
Try different streetType filters:
https://<host>:8083/gateway/odata/SAP/<YOUR_SERVICE>;v=1/StreetSet?$filter=streetType eq ’PEDESTRIAN’
https://<host>:8083/gateway/odata/SAP/<YOUR_SERVICE>;v=1/StreetSet?$filter=streetType eq ‘FOOTWAY’
And also name filters:
https://<host>:8083/gateway/odata/SAP/<YOUR_SERVICE>;v=1/StreetSet?$filter=name eq ‘East Lake Street’
https://<host>:8083/gateway/odata/SAP/<YOUR_SERVICE>;v=1/StreetSet?$filter=name eq ‘North Canal Street’
Try combined filter
https://<host>:8083/gateway/odata/SAP/<YOUR_SERVICE>;v=1/StreetSet?$filter=streetType eq ‘PEDESTRIAN’ and name eq ‘East Lake Street’
There should be different and meaningful response to these calls.
Full source code
import org.apache.olingo.odata2.api.edm.EdmException
import org.apache.olingo.odata2.api.edm.EdmLiteral
import org.apache.olingo.odata2.api.edm.EdmProperty
import org.apache.olingo.odata2.api.edm.EdmTyped
import org.apache.olingo.odata2.api.exception.ODataApplicationException;
import org.apache.olingo.odata2.api.uri.UriInfo
import org.apache.olingo.odata2.api.uri.expression.BinaryExpression
import org.apache.olingo.odata2.api.uri.expression.BinaryOperator
import org.apache.olingo.odata2.api.uri.expression.ExceptionVisitExpression;
import org.apache.olingo.odata2.api.uri.expression.ExpressionVisitor
import org.apache.olingo.odata2.api.uri.expression.FilterExpression
import org.apache.olingo.odata2.api.uri.expression.LiteralExpression
import org.apache.olingo.odata2.api.uri.expression.MemberExpression
import org.apache.olingo.odata2.api.uri.expression.MethodExpression
import org.apache.olingo.odata2.api.uri.expression.MethodOperator
import org.apache.olingo.odata2.api.uri.expression.OrderByExpression
import org.apache.olingo.odata2.api.uri.expression.OrderExpression
import org.apache.olingo.odata2.api.uri.expression.PropertyExpression
import org.apache.olingo.odata2.api.uri.expression.SortOrder
import org.apache.olingo.odata2.api.uri.expression.UnaryExpression
import org.apache.olingo.odata2.api.uri.expression.UnaryOperator
import com.sap.gateway.core.ip.component.commons.ODataExchangeHeaderProperty
import com.sap.gateway.ip.core.customdev.logging.ILogger;
import com.sap.gateway.ip.core.customdev.logging.LogMessage;
import com.sap.gateway.ip.core.customdev.util.Message
/*
* ***************************
* FRAMEWORK CALLBACK METHODS
* ***************************
*/
/** Function processRequestData will be called before the request data is handed over to the REST service and can be used for providing filter capabilities. User can manipulate the request data here.
*/
def Message processRequestData(message) { handleFilter(message); return message;
}
/** Function processResponseData will be called after the response data is received from the REST service and is used to convert the REST response to OData response using String APIs. User can manipulate the response data here.
*/
def Message processResponseData(message) { convertResponse(message); return message;
}
/*
* ***************************
* RESPONSE HANDLING
* ***************************
*/
def void convertResponse(Message message){ String restResponse = message.getBody().toString(); /* CONVERT PAYLOAD */ int index = restResponse.indexOf("<result>"); if(index == -1){ // in case of empty payload restResponse = "<StreetSet><Street><gid></gid></Street></StreetSet>"; // currently the only possibility for returning nothing }else{ restResponse = restResponse.substring(index); restResponse = restResponse.replaceAll("<result>", "<Street>"); restResponse = restResponse.replaceAll("</result>", "</Street>"); restResponse = restResponse.replaceAll("</results>", "</StreetSet>"); restResponse = "<StreetSet>" + restResponse; } message.setBody(restResponse); message.setHeader("Content-Type", new String("xml"));
}
/*
* ***************************
* REQUEST HANDLING
* ***************************
*/
/**
* supporting $filter for one property only
* */
def void handleFilter(Message message){ //retrieve the FilterExpression UriInfo uriInfo = getUriInfo(message); FilterExpression filterExpression = uriInfo.getFilter(); if (filterExpression == null){ // this is the case if the service is invoked without $filter return; } // Invoke the ExpressionVisitor RestExpressionVisitor visitor = new RestExpressionVisitor(); Object restFilterObject = null; try { restFilterObject = filterExpression.accept(visitor); } catch (ExceptionVisitExpression e) { log.logErrors(LogMessage.TechnicalError, "Error while calling accept() on custom ExpressionVisitor"); return; } catch(ODataApplicationException oe){ log.logErrors(LogMessage.TechnicalError, "Error while calling accept() on custom ExpressionVisitor"); return; } if(restFilterObject == null){ log.logErrors(LogMessage.TechnicalError, "Couldn't retrieve the converted filter expression for the backend REST service."); return ; } // add the filter statement (converted to REST-format) String relativeUri = getRelativeUri(message); // the REST service URL without hostname. In our example it is: /street/streetsearch?lat=52.52&lng=13.41 relativeUri = relativeUri + (String)restFilterObject; // we know that it is a String, because we've implemented it in the Visitor below // encode relativeUri = relativeUri.replaceAll(" ", "%20").replaceAll("'","%27"); // Finally, set the newly composed URI setRelativeUri(message, relativeUri);
}
/**
* Implementation of ExpressionVisitor
*
* Supports combination of 2 filters
* e.g.
* https://<host>/gateway/odata/SAP/<YOUR_SERVICE>/StreetSet?$filter=streetType eq ‘PEDESTRIAN’ and name eq ‘East Lake Street’
*
* */
def class RestExpressionVisitor implements ExpressionVisitor{ /* * This implementation converts the OData-like operators (eq) * to operators in the format, that's understood by the REST service * */ @Override public Object visitBinary(BinaryExpression binaryExpression, BinaryOperator operator, Object leftSide, Object rightSide) { String operatorForREST = ""; switch(operator){ case BinaryOperator.EQ: operatorForREST = "="; break; case BinaryOperator.AND: operatorForREST = "&"; break; default: throw new RuntimeException("This filter operator is not supported: " + operator.toUriLiteral()); } return leftSide + operatorForREST + rightSide; } @Override public Object visitFilterExpression(FilterExpression filterExpression, String expressionString, Object expression) { // we need to add & because the filter is not the first URL parameter if(expression instanceof String){ String expressionAsString = (String)expression; if( ! expressionAsString.startsWith("&")){ expressionAsString = "&" + expressionAsString; return expressionAsString; } } } @Override public Object visitLiteral(LiteralExpression literalExpression, EdmLiteral edmLiteral) { return edmLiteral.getLiteral(); } @Override public Object visitMember(MemberExpression arg0, Object arg1, Object arg2) { return null; } @Override public Object visitMethod(MethodExpression arg0, MethodOperator arg1, List<Object> arg2) { return null; } @Override public Object visitOrder(OrderExpression arg0, Object arg1, SortOrder arg2) { return null; } @Override public Object visitOrderByExpression(OrderByExpression arg0, String arg1,List<Object> arg2) { return null; } /* * We need to do a mapping from OData-property-name to REST-parameter-name * */ @Override public Object visitProperty(PropertyExpression propertyExpression, String uriLiteral, EdmTyped edmTyped) { if(! (edmTyped instanceof EdmProperty)){ throw new RuntimeException("Unexpected type"); } EdmProperty edmProperty = (EdmProperty) edmTyped; String propertyName = null; try { propertyName = edmProperty.getName(); } catch (EdmException e) { ((ILogger)log).logErrors(LogMessage.TechnicalError, "EdmException occurred while accessing filter statment property"); throw new RuntimeException("Unable to access name of property", e); } String restParameterName = null; if(propertyName.equals("streetType")){ // in the REST service, the filter parameter is spelled differently than the property name restParameterName = "streettype"; }else{ restParameterName = propertyName; } return restParameterName; // in our example, this will at the end lead to the expression streettype=PEDESTRIAN } @Override public Object visitUnary(UnaryExpression arg0, UnaryOperator arg1,Object arg2) { return null; }
}
/*
* ***************************
* HELPER
* ***************************
*/
def String getRelativeUri(Message message){ Map headers = message.getHeaders(); Object object = headers.get("RelativeUri"); if(object instanceof String){ return (String)object; } return null;
}
def void setRelativeUri(Message message, String newUri){ message.setHeader("RelativeUri", newUri);
}
def UriInfo getUriInfo(Message message){ Map headers = message.getHeaders(); Object object = headers.get(ODataExchangeHeaderProperty.UriInfo.toString()); // need dependency to bundle com.sap.gw.rt.ip.commons.camel-commons if(object instanceof UriInfo){ return (UriInfo)object; } return null;
}
Further Reading
Preparing Eclipse for Groovy scripting:
http://scn.sap.com/docs/DOC-61719
Introduction in REST datasource part 1: Understanding the return structure in xml
http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/10/understanding-rest-data-source-in-integration-gateway-1-query-very-simplified
Introduction in REST datasource part 2: Understanding the return structure in json
http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/11/integration-gateway-understanding-rest-data-source-2-query--json-very-simplified
Introduction in REST datasource part 3: converting xml payload of a REST service for usage in Integration Gateway
http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/12/integration-gateway-understanding-rest-data-source-3-query--xml-standard
Installing SMP toolkit:
http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/08/22/how-to-install-sap-mobile-platform-tools-for-integration-gateway-in-eclipse-kepler
Tutorial for Integration Gateway:
http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/06/10/creating-an-odata-service-based-on-sap-gateway-soap-jdbc-and-jpa-data-sources-ba
Tutorial about scripting in Integration Gateway:
http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/10/17/integration-gateway-using-custom-scripting-to-map-entity-sets-to-data-sources
-> here we have the difference to OData
