Monday, December 17, 2012

Teiid Designer 8.0 Final Released

Teiid Designer 8.0 Final was released last week and is available for download/installation.

Key to this release was to adapting Designer to be compatible with Teiid 8.2 and the change in server philosophy of the JBoss AS7 application server.

Primary drivers for this release:
  • Support for VDB Reuse.
    • This feature enables you to create and deploy VDBs to your server and re-importing these VDBs as JDBC sources to so you can use them as sources in other view models.
  • Integrate Teiid server contents and UI with JBoss/WTP Server view.

  • Upgrading server-related actions to reflect changes in AS7 functionality
For more details, check out What's New and take it for a drive!



Tuesday, December 11, 2012

Teiid 8.3 Alpha1 Posted

The first alpha of 8.3 is now available in our downloads.  This alpha features:
  • TEIID-2253 Multi-source improvements - multi-source planning was significantly improved, which will result in much better plans when using multi-source models. The user may also now configure the multi-source column name, auto-populate the multi-column in the actual metadata, and use non-literal multi-source column values.
  • TEIID-1557 Generated Key Retrieval - JDBC now supports generated key retrieval, see the Client and the Developers Guides for more.
As well as several important fixes and general socket buffering improvements.  There will of course be more to come with this release shortly.

Steve

Tuesday, November 20, 2012

Teiid 8.2 Final Released

Continuing our time boxed release schedule Teiid team is excited to announce availability of Teiid 8.2.Final.

In total there were 120 issues resolved - including 15 new features and over 50 bug fixes. See all the issues on Teiid's JIRA.

This release coincides with the first release of the Teiid Console for Teiid 8.x.  The Teiid Console is a web based administrative tool for configuration and monitoring and is also available from the downloads area. A big thanks to Heiko Braun for his support in developing this.

The Teiid Designer team is making great progress towards the Teiid Designer 8.0 which works with Teiid 8.2.Final. For early releases please see the Designer Downloads.

Teiid 8.2 is still based on JBoss AS 7.1.1  The JBoss AS 7.2 release is still TDB sometime after the rename process. You can help by voting for new name http://jboss.org/vote.  

Here are the feature highlights:
  • TEIID-1923 Teiid Console - A web based administrative console for Teiid.
  • TEIID-2150 Positional procedure defaults - a procedure now may be called positionally with just the leading input parameters as long as all trailing parameters are out or defaultable.
  • TEIID-2148 Streaming Improvements - the ws translator invoke/invokeHttp procedures now accept a "stream"" parameter to indicate that the result should be streamed directly without an intermediate copy being created. Also intermediate document/blob copies are made concurrently with reading to reduce latency.
  • TEIID-2158 Rest Services - when required extension metadata defined on the virtual procedure, a REST based service on that procedure is automatically generated and deployed upon VDB deployment.
  • TEIID-2163 Binary XMLSERIALIZE - XMLSERIALIZE now supports blob/varbinary and the encoding/version and XMLDECLARATION options.
  • TEIID-2141 OBJECTTABLE - was added to extract values from objects using arbitrary scripts. See the Reference for usage.
  • TEIID-2111 Vararg Procedures - variable argument procedures and functions can be created through DDL using the VARIADIC parameter type. See the Reference for usage.
  • TEIID-2195 Function Based Indexes - DDL can be used to specify function based indexes. When used on an internal mat view it will create the corresponding function based index automatically. In any other circumstance, the metadata is not currently used.
  • TEIID-2181 System query performance - system queries are now indexed on case-insensitive schema/table name columns.
  • TEIID-2086 Transactional results caching - the result set cache is now transactional by default.
  • TEIID-2210 Object Translator - the translator-object can support Infinispan Cache and other Map type caches. See OBJECTTABLE for executing queries against this translator.
  • TEIID-1386 Procedure exception handling - the Teiid procedure language now supports exception handling and emitting SQLWarnings. See the Reference for syntax and usage.
  • TEIID-2176,TEIID-2177 Native Query Support - JDBC, LDAP, SalesForce, JPA, OLAP translators now support execution of native queries directly against the source either through "native-query" extension metadata property or native procedure defined by their translator.
  • TEIID-2227 Procedure RETURN statement - the Teiid procedure language now supports a RETURN statement. See the Reference for syntax and usage.
  • TEIID-2269 Vault Support - The transport's keystore and truststore passwords are now supported through vault based passwords.
  • TEIID-1064 Google Spreadsheet Support - there is now support for querying Google Spreadsheets via Teiid. See the Reference and Admin Guides for more.
  • TEIID-2274, TEIID-2275, TEIID-2288, TEIID-2299 Converting examples to Quick Starts - The examples in the Teiid JBoss dist zip have been converted to Quick Starts and will now be found in the Quick Start kit.
This work was due in large part to the great participation from Teiid community users, a BIG thanks to all who participated especially Mark Addleman, Jack Ma, I Leitch, Vineela Gumpa, Samier Nadji, Graeme Gillies, Ivan Chan, David Eichmann, Hisanobu Okuda, Jaspal Singh, Mónica L.A., Rajasekhar Kota, Filip Nguyen in providing valuable input through their JIRAs and community suggestions.

How To Get Involved:

If you have useful features in mind for Teiid, please open a forum discussion and lets get them captured in JIRA for inclusion in the Teiid roadmap.  8.3 is just starting and can still be influenced based upon community need. 

If you are a developer, did you know Teiid source is now hosted on GitHub? Your contribution is just a fork away! 

If you are using Teiid, please blog about it and tell us how it is helping you solve your Data Virtulization needs. If you have/can create tutorials or videos about Teiid, please share with the community.

Thank you.

Ramesh & Steve

Monday, November 5, 2012

Introducing Teiid Admin Web Console - 1.0.CR1

Teiid team is very exited to introduce a new GWT based administrative console for Teiid starting with version 8.2.x

If you are already using slick JBoss AS7 web console, then you already know how good, quick and snappier it is to configure and manage JBoss AS7. Now, you can use the same console to configure and manage Teiid. This new console is an extension to the JBoss AS7 console, written using GWT based technology and uses AS7 DMR underneath for remote communication with Teiid Subsystem on JBoss AS7.

The code for Teiid Admin Web Console is  hosted at https://github.com/teiid/teiid-web-console

You can find some documentation at https://docs.jboss.org/author/display/TEIID/Teiid+Admin+Web+Console



With the new console you can
  • Configure  Teiid
  • Deploy VDBs
  • Check the status of the VDB
  • Check deployment errors of the VDB
  • View models in a VDB and check their properties and and any error associated with them
  • Check data roles, manage role assignments
  • Check current sessions logged against VDB
  • View current requests against a VDB and ability to cancel them
  • View the schema of a VDB
  • View the query-plan of a in-flight query
  • Check and manage the status materialization tables
and much more. Please download the Teiid Web Console and give it a try and give us feedback and what other features you would like to see in this tool future releases. Also if you like GWT programming and would like to contribute to the project please let us know.

Thank you.

Ramesh..

Teiid 8.2 CR1 Posted And More

We're closing in on our next release with Teiid 8.2 CR1 now available for deployment on AS7 7.1.1-Final. New features include: 
  • TEIID-2269 Vault Support - The transport's keystore and truststore passwords are now supported through vault based passwords.
  • TEIID-1064 Google Spreadsheet Support - there is now support for querying Google Spreadsheets via Teiid. See the Reference and Admin Guides for more.
  • TEIID-1241 WSDL backed WS Translator - a web services data source can be configured to access a single server and point based upon a specific WSDL.
In total we have addressed over 100 issues (which is now common for a point release). Please ensure that all of your issues have been addressed satisfactorily as we likely release a CR2 before proceeding to final.  

The new Google spreadsheet support, which was an excellent contribution from Red Hatter's Filip Nguyen and Filip Elias, will hopefully be broadened into other Google services over time.

We're also looking at numerous doc clean-ups especially to our examples and quick starts.  You can also assist with the docs via comments or since document editing moderation is turned on, you can submit changes directly.

And finally if you're using Designer or new to Teiid in general, community member Gioppo Luca has been working on tutorials using the latest Teiid Designer release version 7.8 with Teiid 8.1.

Steve

Tuesday, October 23, 2012

Teiid 8.2 Beta2 Released

Teiid 8.2 Beta2 is released today. We are just about a week away from CR release, that marks the end of development of features for 8.2 release and only bugs will be fixed before 8.2.Final.

This release brings better integration with "object cache" stores like Infinispan. If you have data in Infinispan (local or remote) now you can integrate the objects in the map with other relational or non-relational data. This release also fixes few bugs, please look at the JIRA for further details.

If you are working with Teiid 8.1, this is the time for you to pick up 8.2 version and check to make sure your use cases work with out any issues. If you see any issues please log a JIRA, so that it can be fixed before the 8.2 Final release.

Thank you.

Ramesh..

Friday, October 12, 2012

Teiid 8.2 Beta1 Released

8.2 Beta1 has been posted to the downloads: http://www.jboss.org/teiid/downloads

Teiid 8.2 has added numerous fixes since the last alpha and the additional feature of a RETURN statement to the procedure language. Also the newly added OBJECT translator is still being refined and is not yet ready for use.

Be sure to take a test drive and log any issues found. We should have the next beta available within two weeks before locking things down more for CR and Final releases.

Our transition of main line development from SVN to Git is now complete- https://github.com/teiid/teiid. Let us know if you have any issues getting started with Git as we'll be looking to improve our wiki and process related documentation.

Thanks,
Steve

Also Red Hat Integration week is imminent. If you have not registered, register and join us for many sessions around Red Hat supported middleware integration - many of which involve Teiid!

Friday, September 28, 2012

Teiid 8.2 Alpha2 Released

8.2 Alpha2 has been posted to the downloads: http://www.jboss.org/teiid/downloads 
Teiid 8.2, is beginning to be feature packed release with a long overdue feature of exception handling in virtual procedures. A developer can write code like RAISE SQLEXCEPTION .. and capture that event in the client with a SQLException. Read more about it at https://docs.jboss.org/author/display/TEIID/Procedure+Language 
Native Query support is expanded. What is native query, how is it related? Well a native query any arbitrary command (SQL or otherwise) that is specific to a given source. With this new feature, every Teiid translator provides a mechanism, where you can submit any native command directly to the physical source through translator, and translator will execute and return the results. Why you need it? Well, taking an example of Oracle, lets say you have written a very specialized optimized query and you would like to submit as is with out Teiid re-writing it, then this is way to do that. The usecases are endless when you think about non-relational sources like NOSQL stores. However, this comes with word of caution , that is when using this feature, any abstract nature of your view where the results can be predicted is not maintained through Teiid layers. This also exposes your source directly to anybody would like to issue arbitrary commands, so use this feature with caution or lock-it down with a data role if you do not use it. Read more https://docs.jboss.org/author/display/TEIID/Built-in+Translators 

We are starting to foray into integration with Big data stores. We have released Hive based translator for HDFS in the previous release, in this release we are presenting object/infinispan translator for cache and for source integration. This will be our framework for bringing in object/document based stores into Teiid. This combined with OBJECTTABLE construct introduced in the 8.2 Alpha1 going to be very useful and productive.

Teiid 8.2 Beta1 will be in early October.  Be sure to take a test drive and log any issues found.  There is still plenty of time to get some more small features in, so come to Teiid forums and ask a question.

Redhat is also putting together a Redhat Integration week, if you have not registered, register and join us for many sessions around Redhat supported middleware integration stack, which also involves Teiid! 
 
Thank you.
 
Ramesh..

Friday, September 7, 2012

Teiid on OpenShift demo is available!

We now have a Teiid demo available for deployment to OpenShift.  Teiid allows you to access and integrate data from a variety of distributed data sources - in real-time, without moving your data from its system of record.  OpenShift is Red Hat's Cloud Computing Platform as a Service (PaaS) offering.

You can now deploy the Teiid engine to a free OpenShift cloud instance and try it for yourself!  To get your free demo started today, see our install instructions .

The demo provides easy-to-follow articles which will show you how to access and integrate data from three disparate sources:
  • MySQL database containing account data
  • Twitter feed webservice
  • Salesforce instance
These are just a few of the many sources you can access with Teiid.  And with Teiid, you can easily join any of the sources to create your own federated data views!

We also show you how to access your Teiid-OpenShift VDB (virtual database) in a number of ways: 
  • Submit ad-hoc queries via the provided browser-based query application.
  • Connect via JDBC - with your favorite database or reporting tools.
  • Deploy and access as a REST service.
Want to find out more?  See our install instructions on github to get started!

Tuesday, August 21, 2012

Teiid 8.1 Final Released, Brings back "embedded" Teiid

The Teiid Team is very excited to announce the availability of Teiid 8.1-Final for JBoss AS 7.1.1.  If you are working with Teiid 8.0, we highly recommend to migrate to 8.1-Final version, as we have about 100 issues resolved along with many new features.

In Teiid 6.x versions, Teiid used to support a "embedded" version, this feature has been removed from 7.x versions, now it has been resurrected  with slightly different purpose.

Teiid Embedded

The main purpose of "Teiid Embedded" is to provide a data virtualization framework with full metadata definitions, query planning and query optimizations across multiple disparate sources with full query features minus any platform specific details such as external vdb deployments, connection management, clustering, authentications etc.  This provides a perfect environment to write a very specific data integration and virtualization needs in any vertical market for your application with out any JBoss AS specific dependencies.

To support the "Teiid Embedded", we have added separate downloadable kit, with a fully functional example.  Go ahead download it, and hack it away. BTW, please let us now if there are any features we can add to improve your usecase.  Also check out blog by Andrily on Teiid Embedded

Below are other notable new features in the Teiid 8.1-Final
  • TEIID-1366 VDB Reuse - a vdb.xml can now declare imported vdbs to reuse metadata. This is a huge feature, will do another blog on this very soon.
  • TEIID-2061 Comparable Object - the system property org.teiid.comparableObject can be set to use OBJECT values in comparison/sorting/grouping operations. It is expected that the object values correctly implement Comparable.compareTo.
  • TEIID-2083 Admin Metadata - you can now retrieve metadata in DDL from the admin api via the getSchema method. This works for Designer defined VDBs too. Design a VDB in Designer and deploy it and ask for its metadata in DDL.
  • TEIID-2105 Improved VDB loading - vdb loading logic was refined to make administration easier. ExecutionFactory now has is/setSourceRequiredForMetadata() to indicate whether a source connection is needed for the getMetadata call.
  • TEIID-1598 Translator Result Caching - translators can interact with the result set caching facility via a CacheDirective. See the Developer's Guide for more.
  • TEIID-2077 Result reuse - the engine will automatically detect if the same source query is used multiple times in a plan and reuse the result rather than issuing another query.
  • TEIID-2113 Misc parser improvements - the parser will now accept the LATERAL keyword for defining a LATERAL join (previously we just used the TABLE keyword), unary negation is now supported e.g. -col1, the FOR keyword is optional for TEXTAGG, and the BNF documentation was dramatically improved.
  • TEIID-2036 Dependent Join Array Comparison - platforms supporting array comparisons can have multi-attribute dependent joins pushed down as array comparisons. Oracle, PG, and H2 translators are already marked as supporting array types.

Due to the time constraints and JBoss AS 7.2 time line, Teiid still does not have administrative console GUI web application.  We believe we will provide a initial version of this console in Teiid 8.2 release.

Also, a fully compatible Teiid Designer for Teiid 8.0 is still in works. Until one is available, use Teiid Designer 7.7 to develop your VDBs - however you will not be able be able to administer/preview a Teiid 8.1 server.

Also many thanks all our community users especially Mark Addleman, Andriy Rokhmanov, Manoj Agarwal, Simon Green, Bob Akers, Deshi Xiao, Nabeel Ahmed, Heidi Muehlebach in providing valuable input through their JIRAs and community suggestions.

Guys keep them coming, please help us make Teiid a thriving community that provides solutions in Data Virtualization space.

Thank you.

Ramesh & Steve

Monday, August 13, 2012

Teiid 8.1 CR1 Posted

Teiid 8.1 CR1 is now available for deployment on AS7 7.1.1-Final. We are excited to be closing in on the final release.  New features since Beta1 include: 
  • TEIID-1598 Translator Result Caching - translators can interact with the result set caching facility via a CacheDirective. See the Developer's Guide for more.
  • TEIID-2077 Result reuse - the engine will automatically detect if the same source query is used multiple times in a plan and reuse the result rather than issuing another query.
  • TEIID-2113 Misc parser improvements - the parser will now accept the LATERAL keyword for defining a LATERAL join (previously we just used the TABLE keyword), unary negation is now supported e.g. -col1, the FOR keyword is optional for TEXTAGG, and the BNF documentation was dramatically improved.
  • TEIID-2036 Dependent Join Array Comparison - platforms supporting array comparisons can have multi-attribute dependent joins pushed down as array comparisons. Oracle, PG, and H2 translators are already marked as supporting array types.
We are also closing in on addressing nearly 100 issues, which should ensure a high quality release shortly.

Please ensure that all of your issues have been addressed satisfactorily in the next week and of course log anything new that you find. 

The Teiid Team

Wednesday, August 1, 2012

EDS 5.3 available from Red Hat

If you have been working with Teiid 7.7 version and would like a supported version then look no further than EDS 5.3 from Red Hat.  Red Hat just announced release of their Enterprise Data Services Platform 5.3 which is based on the community projects Teiid and ModeShape and many more.

Some of the key highlights include

  • New data sources: Greenplum, PostgreSQL 9, Apache Hive (tech preview)
  • Performance enhancements
  • New wizards, guides, cheatsheets to enhance tooling usability.
If you are already a subscriber of EDS from Red Hat, then can you download the bits from Red Hat portal now. This also comes with supported tooling JBoss Developer Studio 5.0. Check out new wizards that can help you consume web services or file based sources with ease in EDS.

If you are contemplating on why do you need supported version? well if you are going to deploy a critical application into production environment and need immediate assistance on the product then supported version is the only solution. The product versions are tightly integrated, stable and battle tested on variety of platforms by Red Hat and come with 5 years of support on given version major version. Contact Red Hat sales for more details.

Thank you.

Ramesh..

Monday, July 23, 2012

Teiid 8.1 Beta1

Teiid 8.1 Beta1 is now available for deployment on AS7 7.1.1-Final. New features include: 
  • VDB Reuse a vdb.xml can now declare imported vdbs to reuse metadata.  See the Reference for more.
  • Comparable Object - the system property org.teiid.comparableObject can be set to use OBJECT values in comparison/sorting/grouping operations. It is expected that the object values correctly implement Comparable.compareTo.
  • Admin Metadata - you can now retrieve metadata in DDL from the admin api via the getSchema method.
  • Improved VDB loading - vdb loading logic was refined to make administration easier. ExecutionFactory now has is/setSourceRequiredForMetadata() to indicate whether a source connection is needed for the getMetadata call.
VDB reuse is a powerful feature that enables a lighter-weight approach to sharing schema/models across VDBs.  Design time support will come in a future Designer release.

Overall issue fixing has been brisk with 69 issues addressed so far.  We will likely have another Beta or two before locking down for the CR releases.

Thanks for all the community feedback on the Alphas!

The Teiid Team

Friday, June 29, 2012

Data Solutions Everywhere at Red Hat Summit and JBoss World 2012

A very successful Red Hat Summit and JBoss World 2012 with over 3300 attendees from all over the world happened this week in Boston Hynes Convention Centre. 

In last one week Red Hat has released many products.

There were many varieties of sessions, it was evident that big data and cloud computing has taken prominent spot light. JBoss keynote was so good, there was not even standing room in 800+ capacity session hall. If you have not seen it you can watch it here http://www.youtube.com/watch?v=t1cud4tP4Us&feature=youtu.be

If you have not already taken look at GlusterFS (Red Hat Storage), it is just  magic where you can take commodity hard drives and turn them into big data storage solution all with software.

I have attended couple sessions on Data Grid by Manik, they were very informative in educating the user about Big Data and how Infinispan is solving  issues in this space.

I attended "Red Hat Big Data Strategy Overiew", which show cased Red Hat Enterprise MRG Grid (condor) where they demonstrated, how to manage a Apache Hadoop cluster for job performance and resource utilization. Somebody else at conference also mentioned that this could even manage Red Hat Storage clusters with this and run the Map Reduce jobs on this cluster just like using Hadoop. Very cool!

Justin Hays in his "JBoss enterprise Middleware and BigData" showed how he solved his customer's big data problem with integration of several JBoss Technologies in a minimal amount of time. This was a great case study, hopefully he can do a white paper or do a blog about it.  He used EDS (Teiid) as one of the component. He did release his code in git repo here https://github.com/jboss-tusk/tusk

My presentation went very well, except for some technical difficulties during the demo. It was very well received, there were lot questions from the users ranging from whys to hows. I did felt that lot of developers never heard of data virtualization technologies before or do not fully understand its capabilities. Hopefully with these kind of conferences will help to spread the word more and more. We will also try to post more articles and demos in future.  Please help us spread the word about Data Virtulization. Unfortunately my presentation was not recorded, I will see how I can redo it and record. Mean while download Teiid, take it for spin and ask us any questions you have.

The Teiid team also closely looking into Hibernate OGM and have intentions integrating with NOSQL stores like Infinispan, Cassendra, MangoDB. If you are familiar with these or any other NOSQL stores and want to write Teiid translators please talk to us and help contributing to the community.

IBM sponsored "Summit at Fenway Park", it was lot of fun taking a tour of the Fenway, and stepping on the grounds for batting pose.

Thank you.

Ramesh..

Wednesday, June 20, 2012

Data Virtualization @ Redhat Summit 2012

You have heard about Data Virtualization, but do not completely understand what it is and what benefits it brings to your organization?

Are you still on the fence about Data Virtualization technologies?

Are you considering Data Virtulization technology for your enterprise but do not know what are some essential features you need to be looking out to make it a successful evaluation of technology? Would like to know some common do and don'ts in choosing a vendor?

Would you like to know how other enterprises are using the Data Virtualization technology? What are the problems they are trying to solve?

If you answered "yes" to any one of the above questions, please attend my session "The Power of Data Virtualization" at Redhat Summit 2012

In this session, I will describe what this technology about, and essential features you need to be looking in a Data Virtualization vendor and take you through some usecases that enterprises are using the Data Virtualization technology.

I will also demonstrate an example to show case the JBoss Enterprise Data Services platform an offering from Redhat in Data Virtualization.

If you have not registered for conference, please register for Redhat Summit 2012 and come see us.

Thank You.

Ramesh Reddy.

Wednesday, May 2, 2012

Teiid 8.0 Final Released, brings Data Virtualization to JBoss AS 7.x

The Teiid Team is proud to announce the availability of Teiid 8.0 Final for JBoss AS 7.1.1.

A major effort with this release was moving from JBoss AS 5.1.0 platform to the new JBoss AS 7.1.1 platform.  We super exited with the move as JBoss AS7 offers:
  • Blazing fast start-up, deployment and configuration
  • Modular design, with powerful class loader
  • Exceptionally light weight resource consuption
  • Simple, centralized and consistent configuration
  • Standards compliance
  • Simplified Clustering
  • Infinispan support
  • Easy testing
Read more about JBoss AS 7.1.1 here. This means all the above features/services are also applicable to Teiid. That is not all, we have advanced Teiid 8 with powerful features, performance enhancements, and query optimizations. The Teiid 8 feature list includes:
  • DDL Based View Definitions - Define virtual tables, procedures and functions for Dynamic VDBs using DDL.
  • MetadataRepository - Pluggable metadata facility for any VDB and models inside it.
  • CallableStatement Named Parameters - you can now use CallableStatement named parameter get/set methods.
  • New Translator capabilities
    • translators may indicate which convert functions they support
    • restrict non-join comparisons to only literals.
    • return ReusableExecution instances for processing nodes that issue multiple queries.
    • translators may indicate support for dependent join handling
  • Continuous Asynch Queries to process plans in a streamed window fashion the TeiidStatement/TeiidPreparedStatement methods now take a RequestOptions object to specify continuous mode. See the Client and Developers Guides for more.
  • Texttable selectors - can be used to selectively parse only record lines matching a given selector string. Selectors may also be used for column values to join data from other records positionally.
  • Enhanced Comparison Support - see the Admin Guide more.
    • Comparable LOBs - the system property org.teiid.comparableLobs can be set to use CLOB and BLOB values in comparison/sorting/grouping operations.
    • Padded Comparison - the system property org.teiid.padSpace can be set to effectively right pad strings to the same length for comparison.
    • Collation Support - the system property org.teiid.collationLocale can be set to use a different collation than the Java UTF-16 default.
  • VARBINARY type support - the Teiid VARBINARY type can now be used to support source BINARY and VARBINARY types.
  • Greenplum Translator - for use with the Greenplum database.
  • Enhanced parse/format pushdown - added more built-in support and extension points for parse/format function pushdown. Added parse/format timestamp handling for SQLServer, Sybase, Oracle, and PostgreSQL.
  • User Defined Aggregates - user defined aggregate functions can be defined via extension metadata, DDL, or connector metadata.
  • SET PAYLOAD statement - SET PAYLOAD can be used to set a name value pair on a session scoped payload that will be sent with requests.
  • ENCRYPT REQUESTS - encryptRequests may be used as a connection/datasource property when not using SSL to indicate that request messages and any associated payload should be encrypted.
Teiid 8.0 still lacks an administrative console GUI web application.  We are working to bring you that in the Teiid 8.1 release. Meanwhile JBoss CLI tooling can be used for any configuration purposes.

Also, a fully compatible Teiid Designer for Teiid 8.0 is still in works. Until one is available, use Teiid Designer 7.7 to develop your VDBs - however you will not be able be able to administer/preview a Teiid 8.0 server.

Download Teiid 8.0, take it for a spin and let us know what you think.

Thank you,

The Teiid Team

Friday, April 13, 2012

Teiid 8.0 CR1

We're getting close! Teiid 8.0 CR1 is now available for deployment on AS7 7.1.1-Final.  This release adds the following notable features in addition to numerous defect fixes: 
  • Greenplum Translator - for use with the Greenplum database.
  • Enhanced parse/format pushdown - added more built-in support and extension points for parse/format function pushdown. Added parse/format timestamp handling for SQLServer, Sybase, Oracle, and PostgreSQL.
  • User Defined Aggregates - user defined aggregate functions can be defined via extension metadata, DDL, or connector metadata.  
8.1 development will now start in earnest with only high priority fixes still to be addressed for 8.0.  Be sure to log a JIRA is you see something before the final release is cut.

There will be more on user defined aggregates in another post.

The Teiid Team


Teiid User Defined Aggregate Support

Building upon our support for pushdown and non-pushdown user defined scalar functions, Teiid with 8.0 CR1 also offers the ability to specify user defined aggregate functions (UDAF).

UDAFs are useful to extend Teiid's already extensive aggregate function library.  They also allow Teiid to implement the final processing over pushdown aggregation. For partitioned datasets there is no additional work for Teiid to do, but the UDAF may be needed to express custom aggregation to the participating data sources. For non-partitioned datasets single argument UDAFs may be marked as decomposable to indicate that not only can the initial aggregation be pushed down, but that Teiid will perform the final integration/aggregation of results itself. This provides map-reduce like behavior where processing can be performed in parallel on a large number of source systems.

UDAFs are defined similarly to UDFs and share most of the same metadata properties.  Building on the prior post using Dynamic VDB DDL metadata we can add a user defined aggregate to mimic the commonly supported GROUP_CONCAT aggregate function:

dynamic-portfolio-vdb.xml

 <vdb name="DynamicPortfolio" version= "1">  
   <model name="MarketData">  
     <source name="text-connector" translator-name="file" connection-jndi-name="java:/marketdata-file"/>  
   </model>  
   <model visible = "true" type = "VIRTUAL" name = "portfolio">  
      <metadata type = "DDL"><![CDATA[  
        CREATE VIEW stock (  
         symbol varchar,  
         price decimal  
         ) AS   
          select stock.* from (call MarketData.getTextFiles('*.txt')) f,   
          TEXTTABLE(f.file COLUMNS symbol string, price bigdecimal HEADER) stock;

        CREATE VIRTUAL FUNCTION GROUP_CONCAT(val STRING, sep CHAR) RETURNS STRING 
          OPTIONS (AGGREGATE 'TRUE', "NULL-ON-NULL" 'TRUE', JAVA_CLASS 'example.GroupConcat',
            "ALLOWS-ORDERBY" 'TRUE', JAVA_METHOD 'addInput');
      ]]>  
      </metadata>  
   </model>  
 </vdb> 

The Java class shown below to back the non-pushdown function is straight-forward. The class, designated by the JAVA_CLASS option, must extend org.teiid.UserDefinedAggregate and provide a method, designated by the JAVA_METHOD option, with a signature that accepts the expected inputs.


public static class GroupConcat implements UserDefinedAggregate {
  
 private boolean first = true;
 private boolean isNull = true;
 private StringBuffer buffer = new StringBuffer();
  
 public void addInput(String val, char separator) {
  if (!first) {
   buffer.append(separator);
  }
  buffer.append(val);
  first = false;
  isNull = false;
 }
  
 @Override
 public String getResult(org.teiid.CommandContext commandContext) {
  if (isNull) {
   return null;
  }
  return buffer.toString();
 }

 @Override
 public void reset() {
  first = true;
  isNull = true;
  buffer = new StringBuffer();
 } 
}

You can follow the instructions in the Developer's Guide for more on coding and deploying user defined functions. With the VDB and the UDAF module deployed you should now be able to use the aggregate in your queries.

For example:
SELECT GROUP_CONCAT(SYMBOL, ',' ORDER BY PRICE) AS STOCK_LIST FROM STOCK

For now though this feature is for leading edge users as Teiid Designer support is still unfolding.  Designer 8.0 will overhaul their usage of the Function model which has traditionally been used for UDFs.  So there will be more to come from both the Teiid and Teiid Designer sides shortly.    

Steve

Wednesday, April 11, 2012

Teiid Designer 7.7 Released

The Teiid Designer project team is proud to announce it's 7.7 release is now available for download.

New Teiid Designer Guides View
This release has focused primarily on usability improvements which include:
  • New Guides View
    • Simplifies primary modeling use-cases
  • New Status View
    • Runtime project analysis and status
  • New Designer Cheat Sheets
  • Improved Web Service Importer
    • Results in single queryable view procedure
    • Go from WSDL to testing your VDB in minutes
  • Improved View Table Creation


(See Teiid Designer 7.7 What's New for details and more screen shots)

Tuesday, April 3, 2012

Domain Mode Aware Teiid

Teiid 8.0.0.Beta2 is released today with  "domain mode" support provided by JBoss AS 7. This release is based on JBoss AS 7.1.1

So, what does "domain mode" provide to Teiid?

When you have more than one JBoss AS instance in your server farm and if they are started in domain mode, all the configuration options of this server farm can be centrally managed. For example, you can deploy a VDB or create a data source across all the instances, with one single CLI based call.  Teiid extends this configuration concept to deploy the VDBs, Translators across the whole server farm.

Also, when domain mode is combined with "HA (high availability)" profile, one can cluster the JBoss AS server instances that are deployed. Teiid chooses the HA profile as default profile in its "domain-teiid.xml" file. When the Server is started using the "domain-teiid.xml" the distributed caching that is used for ResultSet caching and Internal Materialized caching is automatically configured. The usage of Admin API is same in both standalone mode and domain mode. See the following document to start in Domain Mode with cluster capabilities.

When multiple Teiid instances are available in a cluster, you can make use load balancing and fail-over features. Check out Teiid documents.

Other Stuff

We are fast approaching CR1 release in next couple weeks, then soon followed by  8.0.0-Final, so check your use cases and let us know if you see any issues before it is too late to get them into 8.0.0-Final.

We are also planning the 8.1 work currently, so if you have any feature(s) you think that Teiid should implement speak up and open a dialogue in Teiid forums.

We are looking for GWT experts to help us with Teiid Console application in 8.1, so if you got some time to spare and want to contribute back to Teiid, please contact us.

Download and try it out and let us know if you see any issues or question in Teiid forums.

Thank you.

Ramesh.. 

Friday, March 9, 2012

Dynamic VDBs Are Back With A Punch!!!

You did not think we removed this feature from Teiid forever did you? In earlier releases of Teiid 8.0 cycles, Dynamic VDBs were not supported due to lack of non-archive file based deployment support from the JBoss AS 7. Now that  JBoss AS 7.1.0-Final has been released they added much needed feature to enable the Dynamic VDBs. With that we are announcing the first release of Teiid that is based on JBoss AS 7.1.0-Final.

Teiid 8.0.Beta1 is released today. Teiid 8.0.Final will also be based upon same version of JBoss AS. This release brings back Dynamic VDB feature, but that is not the big news. Now you can define virtual tables, procedures and functions as you did in Teiid Designer based VDB are now can be defined in the Dynamic VDBs using DDL with out Teiid Designer. Yes, using DDL! This can lead design of Virtual Databases that are more dynamic in nature that can be changed programatically. Without any more non-sense here is quick example.

The below example is extension of the example from Quick Start Guide, so if you are new to Teiid start with that example. Also a full project with all files is also available in the Teiid download archive in the examples section (see docs/teiid/examples/dynamicvdb-portfolio)

dynamic-portfolio-vdb.xml
 <vdb name="DynamicPortfolio" version= "1">  
   <model name="MarketData">  
     <source name="text-connector" translator-name="file" connection-jndi-name="java:/marketdata-file"/>  
   </model>  
   <model visible = "true" type = "VIRTUAL" name = "portfolio">  
      <metadata import-type = "DDL"><![CDATA[  
        CREATE VIEW stock (  
         symbol varchar,  
         price decimal  
         ) AS   
          select stock.* from (call MarketData.getTextFiles('*.txt')) f,   
          TEXTTABLE(f.file COLUMNS symbol string, price bigdecimal HEADER) stock;  
      ]]>  
      </metadata>  
   </model>  
 </vdb>  

Note in the "portfolio" model, we created a VIEW (stock) that is based on the procedure "getTextFiles", which is exposed by a "file" translator. What we effectively have done here is turned some data from CSV file into a table using "TEXTTABLE", and exposed those results as a table. In prior Teiid 7.x releases you could have defined the "TEXTTABLE" construct as shown in view query above after "AS" keyword, however there was no facility to define an abstraction (or logical) view to expose the results of TEXTTABLE. So, when you deploy the above VDB, you can issue queries like

 SELECT * FROM stock WHERE price > 50.00  

using your JDBC connection to the Teiid's Virtual Database. This effectively created an abstraction for your end users, to hide the real details behind your data integration. That is one of basic tenants of data virtualization.

Read related documentation on this feature at https://docs.jboss.org/author/display/TEIID/View+Definitions+in+VDB

Download and try it out and let us know if you see any issues or question in Teiid forums.

Thank you.

Ramesh.. 

Monday, February 27, 2012

Teiid 7.7 Final Released

We are pleased to announce Teiid 7.7 is now available.  While mostly a fix release with nearly 50 issues addressed, highlights of 7.7 features include:
  • Excel JDBC Translator - for use with Excel using the JDBC-ODBC bridge.
  • Salesforce Aggregates - salesforce pushdown queries now support GROUP BY, HAVING, and the standard aggregate functions.
  • Comparable LOBs - the system property org.teiid.comparableLobs can be set to use CLOB and BLOB values in comparison/sorting/grouping operations.
  • Padded String Comparison - the system property org.teiid.padSpace can be set to effectively right pad strings to the same length for comparison.
See the release notes for more details.  Download today, give it try, and vote for or log features you want in upcoming releases.

Steve

Tuesday, February 14, 2012

8.0.0 alpha2

Second instalment in 8.0 series is now available as 8.0.0.alpha2 release. This release is designed to be deployed on AS7 7.1.0-CR1b.

The main changes include in this release are

- New data type, VARBINARY has been added.
- Documentation moved to confluence
- Enhanced comparison support, includes comparable lobs, padded string comparison support and collation support.
- Event ids along with all logged messages
- many more bug fixes.

Dynamic VDBs are still not supported, however once the AS 7.1.0-Final is released this will be supported.

Test it out and let us know if any features not working as you expected or any new features you would like to see in 8.0-Final release as the window for new features is narrowing.

Thank you community for the valuable input. Keep em coming!

Ramesh..

Monday, February 13, 2012

Teiid 7.7 CR1 Posted

7.7 CR1 is now available.  7.7 is becoming a more feature-full release, highlights include:

  • Excel JDBC Translator - for use with Excel using the JDBC-ODBC bridge.
  • Salesforce Aggregates - salesforce pushdown queries now support GROUP BY, HAVING, and the standard aggregate functions.
  • Comparable LOBs - the system property org.teiid.comparableLobs can be set to use CLOB and BLOB values in comparison/sorting/grouping operations.
  • Padded String Comparison - the system property org.teiid.padSpace can be set to effectively right pad strings to the same length for comparison.
Over 40 issues have been addressed in total.  Please validate any features or fixes of interest before the final release.

The Teiid Team

Thursday, January 5, 2012

Teiid 8.0.alpha1 - JBoss AS7 based Teiid

Teiid Team is proud to announce the availability of Teiid 8.0-alpha1 release. This is the first release of Teiid that is based on JBoss AS 7 as host environment. It is also start of the next major revision for Teiid and departure from Teiid 7.x releases. There will not be any more 7.x based releases. There is still lot more work to do for 8.0 Final release, this is our effort to get the community involved early on and weed out any possible regressions.

There are many things changed with respect to AS7, as it completely re-written and entirely different from JBoss 5.x and 6.x releases. For details please see the AS7 documentation. That means there are numerous changes tricked down for Teiid too. I will list some issues to look out for

  • All the deprecated features from the 7.x release have been removed.
  • AS7 provides completely new way of configuration. Teiid utilizes this new configuration mechanism. Fully Supports CLI based administration.
  • Deployment of artifacts such as vdbs and data sources is different from 7.x releases
  • Class loading in AS7 is modularized, that means no more copying of your jdbc jar files into "lib" directory and expect to work. 
  • We are in process of updating the documents to reflect changes (we have done some).
  • Dynamic VDB is not working yet. It will be fixed before final (AS7-1920)
  • Designer is not yet available for 8.0.0.alpha1 release, however you can still use the older Designer to build the VDB. The preview functionality will not work with new server yet. (This will be coming soon..)
  • No "admin-console" application available yet for Teiid. If you can contribute for this please talk to us. We are looking for developers with GWT experience.
If you have VDBs from previous release, they work as is, if you were not using any deprecated features. There is a change as to how you deploy the VDB and how you create the data sources to be used by the VDB. There is absolutely no change in how you access the data, that is JDBC interface.

Excited? if you want to take it for test drive,
  1. Download the AS7-CR1 release and install it by unzipping it
  2. Download the Teiid 8.0.alpha1 and overly the zip file on AS7 install directory
Then start the Teiid in "standalone" mode

${as7-install}/bin/standalone.sh --server-config=standalone-teiid.xml

to start in the "domain" mode

${as7-install}/bin/domain.sh -c domain-teiid.xml --host-config=host-teiid.xml

Surprised? What was your boot time? This is just a beginning. We plan to write couple blogs during this 8.0 release that introduces to all advantages of the new AS7 + Teiid 8.0 platform. Stay tuned.


Thank you.

The Teiid Team.