Teiid 8.0 Final Released, brings Data Virtualization to JBoss AS 7.x

The Teiid Team is proud to announce the availability of Teiid 8.0 Final for JBoss AS 7.1.1.

A major effort with this release was moving from JBoss AS 5.1.0 platform to the new JBoss AS 7.1.1 platform.  We super exited with the move as JBoss AS7 offers:

• Blazing fast start-up, deployment and configuration
• Modular design, with powerful class loader
• Exceptionally light weight resource consuption
• Simple, centralized and consistent configuration
• Standards compliance
• Simplified Clustering
• Infinispan support
• Easy testing

Read more about JBoss AS 7.1.1 here. This means all the above features/services are also applicable to Teiid. That is not all, we have advanced Teiid 8 with powerful features, performance enhancements, and query optimizations. The Teiid 8 feature list includes:

• DDL Based View Definitions - Define virtual tables, procedures and functions for Dynamic VDBs using DDL.
• MetadataRepository - Pluggable metadata facility for any VDB and models inside it.
• CallableStatement Named Parameters - you can now use CallableStatement named parameter get/set methods.
• New Translator capabilities
  • translators may indicate which convert functions they support
  • restrict non-join comparisons to only literals.
  • return ReusableExecution instances for processing nodes that issue multiple queries.
  • translators may indicate support for dependent join handling
• Continuous Asynch Queries to process plans in a streamed window fashion the TeiidStatement/TeiidPreparedStatement methods now take a RequestOptions object to specify continuous mode. See the Client and Developers Guides for more.
• Texttable selectors - can be used to selectively parse only record lines matching a given selector string. Selectors may also be used for column values to join data from other records positionally.
• Enhanced Comparison Support - see the Admin Guide more.
  • Comparable LOBs - the system property org.teiid.comparableLobs can be set to use CLOB and BLOB values in comparison/sorting/grouping operations.
  • Padded Comparison - the system property org.teiid.padSpace can be set to effectively right pad strings to the same length for comparison.
  • Collation Support - the system property org.teiid.collationLocale can be set to use a different collation than the Java UTF-16 default.
• VARBINARY type support - the Teiid VARBINARY type can now be used to support source BINARY and VARBINARY types.
• Greenplum Translator - for use with the Greenplum database.
• Enhanced parse/format pushdown - added more built-in support and extension points for parse/format function pushdown. Added parse/format timestamp handling for SQLServer, Sybase, Oracle, and PostgreSQL.
• User Defined Aggregates - user defined aggregate functions can be defined via extension metadata, DDL, or connector metadata.
• SET PAYLOAD statement - SET PAYLOAD can be used to set a name value pair on a session scoped payload that will be sent with requests.
• ENCRYPT REQUESTS - encryptRequests may be used as a connection/datasource property when not using SSL to indicate that request messages and any associated payload should be encrypted.

Teiid 8.0 still lacks an administrative console GUI web application.  We are working to bring you that in the Teiid 8.1 release. Meanwhile JBoss CLI tooling can be used for any configuration purposes.

Also, a fully compatible Teiid Designer for Teiid 8.0 is still in works. Until one is available, use Teiid Designer 7.7 to develop your VDBs - however you will not be able be able to administer/preview a Teiid 8.0 server.

Download Teiid 8.0, take it for a spin and let us know what you think.

Thank you,

The Teiid Team

Teiid 8.0 CR1

We're getting close! Teiid 8.0 CR1 is now available for deployment on AS7 7.1.1-Final.  This release adds the following notable features in addition to numerous defect fixes: 

• Greenplum Translator - for use with the Greenplum database.
• Enhanced parse/format pushdown - added more built-in support and extension points for parse/format function pushdown. Added parse/format timestamp handling for SQLServer, Sybase, Oracle, and PostgreSQL.
• User Defined Aggregates - user defined aggregate functions can be defined via extension metadata, DDL, or connector metadata.  

8.1 development will now start in earnest with only high priority fixes still to be addressed for 8.0.  Be sure to log a JIRA is you see something before the final release is cut.

There will be more on user defined aggregates in another post.

The Teiid Team

Teiid User Defined Aggregate Support

Building upon our support for pushdown and non-pushdown user defined scalar functions, Teiid with 8.0 CR1 also offers the ability to specify user defined aggregate functions (UDAF).

UDAFs are useful to extend Teiid's already extensive aggregate function library.  They also allow Teiid to implement the final processing over pushdown aggregation. For partitioned datasets there is no additional work for Teiid to do, but the UDAF may be needed to express custom aggregation to the participating data sources. For non-partitioned datasets single argument UDAFs may be marked as decomposable to indicate that not only can the initial aggregation be pushed down, but that Teiid will perform the final integration/aggregation of results itself. This provides map-reduce like behavior where processing can be performed in parallel on a large number of source systems.

UDAFs are defined similarly to UDFs and share most of the same metadata properties.  Building on the prior post using Dynamic VDB DDL metadata we can add a user defined aggregate to mimic the commonly supported GROUP_CONCAT aggregate function:

dynamic-portfolio-vdb.xml

 <vdb name="DynamicPortfolio" version= "1">  
   <model name="MarketData">  
     <source name="text-connector" translator-name="file" connection-jndi-name="java:/marketdata-file"/>  
   </model>  
   <model visible = "true" type = "VIRTUAL" name = "portfolio">  
      <metadata import-type = "DDL"><![CDATA[  
        CREATE VIEW stock (  
         symbol varchar,  
         price decimal  
         ) AS   
          select stock.* from (call MarketData.getTextFiles('*.txt')) f,   
          TEXTTABLE(f.file COLUMNS symbol string, price bigdecimal HEADER) stock;

        CREATE VIRTUAL FUNCTION GROUP_CONCAT(val STRING, sep CHAR) RETURNS STRING 
          OPTIONS (AGGREGATE 'TRUE', "NULL-ON-NULL" 'TRUE', JAVA_CLASS 'example.GroupConcat',
            "ALLOWS-ORDERBY" 'TRUE', JAVA_METHOD 'addInput');
      ]]>  
      </metadata>  
   </model>  
 </vdb> 

The Java class shown below to back the non-pushdown function is straight-forward. The class, designated by the JAVA_CLASS option, must extend org.teiid.UserDefinedAggregate and provide a method, designated by the JAVA_METHOD option, with a signature that accepts the expected inputs.

public static class GroupConcat implements UserDefinedAggregate {
  
 private boolean first = true;
 private boolean isNull = true;
 private StringBuffer buffer = new StringBuffer();
  
 public void addInput(String val, char separator) {
  if (!first) {
   buffer.append(separator);
  }
  buffer.append(val);
  first = false;
  isNull = false;
 }
  
 @Override
 public String getResult(org.teiid.CommandContext commandContext) {
  if (isNull) {
   return null;
  }
  return buffer.toString();
 }

 @Override
 public void reset() {
  first = true;
  isNull = true;
  buffer = new StringBuffer();
 } 
}

You can follow the instructions in the Developer's Guide for more on coding and deploying user defined functions. With the VDB and the UDAF module deployed you should now be able to use the aggregate in your queries.

For example:

SELECT GROUP_CONCAT(SYMBOL, ',' ORDER BY PRICE) AS STOCK_LIST FROM STOCK

For now though this feature is for leading edge users as Teiid Designer support is still unfolding.  Designer 8.0 will overhaul their usage of the Function model which has traditionally been used for UDFs.  So there will be more to come from both the Teiid and Teiid Designer sides shortly.    

Steve

Teiid Designer 7.7 Released

The Teiid Designer project team is proud to announce it's 7.7 release is now available for download.

New Teiid Designer Guides View

This release has focused primarily on usability improvements which include:

• New Guides View
• Simplifies primary modeling use-cases
• New Status View
• Runtime project analysis and status
• New Designer Cheat Sheets
• Improved Web Service Importer
• Results in single queryable view procedure
• Go from WSDL to testing your VDB in minutes
• Improved View Table Creation
  

(See Teiid Designer 7.7 What's New for details and more screen shots)

Domain Mode Aware Teiid

Teiid 8.0.0.Beta2 is released today with  "domain mode" support provided by JBoss AS 7. This release is based on JBoss AS 7.1.1

So, what does "domain mode" provide to Teiid?

When you have more than one JBoss AS instance in your server farm and if they are started in domain mode, all the configuration options of this server farm can be centrally managed. For example, you can deploy a VDB or create a data source across all the instances, with one single CLI based call.  Teiid extends this configuration concept to deploy the VDBs, Translators across the whole server farm.

Also, when domain mode is combined with "HA (high availability)" profile, one can cluster the JBoss AS server instances that are deployed. Teiid chooses the HA profile as default profile in its "domain-teiid.xml" file. When the Server is started using the "domain-teiid.xml" the distributed caching that is used for ResultSet caching and Internal Materialized caching is automatically configured. The usage of Admin API is same in both standalone mode and domain mode. See the following document to start in Domain Mode with cluster capabilities.

When multiple Teiid instances are available in a cluster, you can make use load balancing and fail-over features. Check out Teiid documents.

Other Stuff

We are fast approaching CR1 release in next couple weeks, then soon followed by  8.0.0-Final, so check your use cases and let us know if you see any issues before it is too late to get them into 8.0.0-Final.

We are also planning the 8.1 work currently, so if you have any feature(s) you think that Teiid should implement speak up and open a dialogue in Teiid forums.

We are looking for GWT experts to help us with Teiid Console application in 8.1, so if you got some time to spare and want to contribute back to Teiid, please contact us.

Download and try it out and let us know if you see any issues or question in Teiid forums.

Thank you.

Ramesh.. 

Dynamic VDBs Are Back With A Punch!!!

You did not think we removed this feature from Teiid forever did you? In earlier releases of Teiid 8.0 cycles, Dynamic VDBs were not supported due to lack of non-archive file based deployment support from the JBoss AS 7. Now that  JBoss AS 7.1.0-Final has been released they added much needed feature to enable the Dynamic VDBs. With that we are announcing the first release of Teiid that is based on JBoss AS 7.1.0-Final.

Teiid 8.0.Beta1 is released today. Teiid 8.0.Final will also be based upon same version of JBoss AS. This release brings back Dynamic VDB feature, but that is not the big news. Now you can define virtual tables, procedures and functions as you did in Teiid Designer based VDB are now can be defined in the Dynamic VDBs using DDL with out Teiid Designer. Yes, using DDL! This can lead design of Virtual Databases that are more dynamic in nature that can be changed programatically. Without any more non-sense here is quick example.

The below example is extension of the example from Quick Start Guide, so if you are new to Teiid start with that example. Also a full project with all files is also available in the Teiid download archive in the examples section (see docs/teiid/examples/dynamicvdb-portfolio)

dynamic-portfolio-vdb.xml

 <vdb name="DynamicPortfolio" version= "1">  
   <model name="MarketData">  
     <source name="text-connector" translator-name="file" connection-jndi-name="java:/marketdata-file"/>  
   </model>  
   <model visible = "true" type = "VIRTUAL" name = "portfolio">  
      <metadata import-type = "DDL"><![CDATA[  
        CREATE VIEW stock (  
         symbol varchar,  
         price decimal  
         ) AS   
          select stock.* from (call MarketData.getTextFiles('*.txt')) f,   
          TEXTTABLE(f.file COLUMNS symbol string, price bigdecimal HEADER) stock;  
      ]]>  
      </metadata>  
   </model>  
 </vdb>  

Note in the "portfolio" model, we created a VIEW (stock) that is based on the procedure "getTextFiles", which is exposed by a "file" translator. What we effectively have done here is turned some data from CSV file into a table using "TEXTTABLE", and exposed those results as a table. In prior Teiid 7.x releases you could have defined the "TEXTTABLE" construct as shown in view query above after "AS" keyword, however there was no facility to define an abstraction (or logical) view to expose the results of TEXTTABLE. So, when you deploy the above VDB, you can issue queries like

 SELECT * FROM stock WHERE price > 50.00  

using your JDBC connection to the Teiid's Virtual Database. This effectively created an abstraction for your end users, to hide the real details behind your data integration. That is one of basic tenants of data virtualization.

Read related documentation on this feature at https://docs.jboss.org/author/display/TEIID/View+Definitions+in+VDB

Download and try it out and let us know if you see any issues or question in Teiid forums.

Thank you.

Ramesh.. 

Teiid 7.7 Final Released

We are pleased to announce Teiid 7.7 is now available.  While mostly a fix release with nearly 50 issues addressed, highlights of 7.7 features include:

• Excel JDBC Translator - for use with Excel using the JDBC-ODBC bridge.
• Salesforce Aggregates - salesforce pushdown queries now support GROUP BY, HAVING, and the standard aggregate functions.
• Comparable LOBs - the system property org.teiid.comparableLobs can be set to use CLOB and BLOB values in comparison/sorting/grouping operations.
• Padded String Comparison - the system property org.teiid.padSpace can be set to effectively right pad strings to the same length for comparison.

See the release notes for more details.  Download today, give it try, and vote for or log features you want in upcoming releases.

Steve