Friday, April 13, 2012

Teiid 8.0 CR1

We're getting close! Teiid 8.0 CR1 is now available for deployment on AS7 7.1.1-Final.  This release adds the following notable features in addition to numerous defect fixes: 
  • Greenplum Translator - for use with the Greenplum database.
  • Enhanced parse/format pushdown - added more built-in support and extension points for parse/format function pushdown. Added parse/format timestamp handling for SQLServer, Sybase, Oracle, and PostgreSQL.
  • User Defined Aggregates - user defined aggregate functions can be defined via extension metadata, DDL, or connector metadata.  
8.1 development will now start in earnest with only high priority fixes still to be addressed for 8.0.  Be sure to log a JIRA is you see something before the final release is cut.

There will be more on user defined aggregates in another post.

The Teiid Team


Teiid User Defined Aggregate Support

Building upon our support for pushdown and non-pushdown user defined scalar functions, Teiid with 8.0 CR1 also offers the ability to specify user defined aggregate functions (UDAF).

UDAFs are useful to extend Teiid's already extensive aggregate function library.  They also allow Teiid to implement the final processing over pushdown aggregation. For partitioned datasets there is no additional work for Teiid to do, but the UDAF may be needed to express custom aggregation to the participating data sources. For non-partitioned datasets single argument UDAFs may be marked as decomposable to indicate that not only can the initial aggregation be pushed down, but that Teiid will perform the final integration/aggregation of results itself. This provides map-reduce like behavior where processing can be performed in parallel on a large number of source systems.

UDAFs are defined similarly to UDFs and share most of the same metadata properties.  Building on the prior post using Dynamic VDB DDL metadata we can add a user defined aggregate to mimic the commonly supported GROUP_CONCAT aggregate function:

dynamic-portfolio-vdb.xml

 <vdb name="DynamicPortfolio" version= "1">  
   <model name="MarketData">  
     <source name="text-connector" translator-name="file" connection-jndi-name="java:/marketdata-file"/>  
   </model>  
   <model visible = "true" type = "VIRTUAL" name = "portfolio">  
      <metadata type = "DDL"><![CDATA[  
        CREATE VIEW stock (  
         symbol varchar,  
         price decimal  
         ) AS   
          select stock.* from (call MarketData.getTextFiles('*.txt')) f,   
          TEXTTABLE(f.file COLUMNS symbol string, price bigdecimal HEADER) stock;

        CREATE VIRTUAL FUNCTION GROUP_CONCAT(val STRING, sep CHAR) RETURNS STRING 
          OPTIONS (AGGREGATE 'TRUE', "NULL-ON-NULL" 'TRUE', JAVA_CLASS 'example.GroupConcat',
            "ALLOWS-ORDERBY" 'TRUE', JAVA_METHOD 'addInput');
      ]]>  
      </metadata>  
   </model>  
 </vdb> 

The Java class shown below to back the non-pushdown function is straight-forward. The class, designated by the JAVA_CLASS option, must extend org.teiid.UserDefinedAggregate and provide a method, designated by the JAVA_METHOD option, with a signature that accepts the expected inputs.


public static class GroupConcat implements UserDefinedAggregate {
  
 private boolean first = true;
 private boolean isNull = true;
 private StringBuffer buffer = new StringBuffer();
  
 public void addInput(String val, char separator) {
  if (!first) {
   buffer.append(separator);
  }
  buffer.append(val);
  first = false;
  isNull = false;
 }
  
 @Override
 public String getResult(org.teiid.CommandContext commandContext) {
  if (isNull) {
   return null;
  }
  return buffer.toString();
 }

 @Override
 public void reset() {
  first = true;
  isNull = true;
  buffer = new StringBuffer();
 } 
}

You can follow the instructions in the Developer's Guide for more on coding and deploying user defined functions. With the VDB and the UDAF module deployed you should now be able to use the aggregate in your queries.

For example:
SELECT GROUP_CONCAT(SYMBOL, ',' ORDER BY PRICE) AS STOCK_LIST FROM STOCK

For now though this feature is for leading edge users as Teiid Designer support is still unfolding.  Designer 8.0 will overhaul their usage of the Function model which has traditionally been used for UDFs.  So there will be more to come from both the Teiid and Teiid Designer sides shortly.    

Steve

Wednesday, April 11, 2012

Teiid Designer 7.7 Released

The Teiid Designer project team is proud to announce it's 7.7 release is now available for download.

New Teiid Designer Guides View
This release has focused primarily on usability improvements which include:
  • New Guides View
    • Simplifies primary modeling use-cases
  • New Status View
    • Runtime project analysis and status
  • New Designer Cheat Sheets
  • Improved Web Service Importer
    • Results in single queryable view procedure
    • Go from WSDL to testing your VDB in minutes
  • Improved View Table Creation


(See Teiid Designer 7.7 What's New for details and more screen shots)

Tuesday, April 3, 2012

Domain Mode Aware Teiid

Teiid 8.0.0.Beta2 is released today with  "domain mode" support provided by JBoss AS 7. This release is based on JBoss AS 7.1.1

So, what does "domain mode" provide to Teiid?

When you have more than one JBoss AS instance in your server farm and if they are started in domain mode, all the configuration options of this server farm can be centrally managed. For example, you can deploy a VDB or create a data source across all the instances, with one single CLI based call.  Teiid extends this configuration concept to deploy the VDBs, Translators across the whole server farm.

Also, when domain mode is combined with "HA (high availability)" profile, one can cluster the JBoss AS server instances that are deployed. Teiid chooses the HA profile as default profile in its "domain-teiid.xml" file. When the Server is started using the "domain-teiid.xml" the distributed caching that is used for ResultSet caching and Internal Materialized caching is automatically configured. The usage of Admin API is same in both standalone mode and domain mode. See the following document to start in Domain Mode with cluster capabilities.

When multiple Teiid instances are available in a cluster, you can make use load balancing and fail-over features. Check out Teiid documents.

Other Stuff

We are fast approaching CR1 release in next couple weeks, then soon followed by  8.0.0-Final, so check your use cases and let us know if you see any issues before it is too late to get them into 8.0.0-Final.

We are also planning the 8.1 work currently, so if you have any feature(s) you think that Teiid should implement speak up and open a dialogue in Teiid forums.

We are looking for GWT experts to help us with Teiid Console application in 8.1, so if you got some time to spare and want to contribute back to Teiid, please contact us.

Download and try it out and let us know if you see any issues or question in Teiid forums.

Thank you.

Ramesh..