Friday, December 24, 2010

Teiid 7.3 Alpha 2

7.3 Alpha 2 adds partition-wise optimizations, Salesforce fixes, and the ability to cache results with LOBs. As always there are numerous fixes and minor improvements. See the release notes and updated guides for more.

Happy Holidays,
The Teiid Team

Tuesday, December 14, 2010

Teiid 7.3 Alpha 1

7.3 Alpha 1 adds subquery optimization, better support for updatable views, translator specific functions and UDFs, the unescape function to support common escape sequences in strings, and additional predicate optimization. As always there are numerous fixes and minor improvements.

The Teiid Team

Friday, November 5, 2010

Teiid 7.2 Final Released

We are pleased to announce Teiid 7.2 with numerous features, improvements, and bug fixes. Here are some highlights:
  • SQL Support
    • Non-Recursive Common Table Expressions - The WITH clause (and associated pushdown) for non-recursive queries is now supported.
    • Explicit Table Syntax - TABLE x can now be used as a shortcut for SELECT * FROM x
  • JSON Support - JSON, which could be obtained through the ws-translator using http calls or other methods, can be converted to XML using the system function jsonToXml. XMLTABLE and other integration logic can then be used on the JSON results.
  • Transaction Statements - JDBC/ODBC now accepts START TRANSACTION, COMMIT, and ROLLBACK statements to control local transactions.
  • Procedure Result Caching - virtual procedure definitions may use a cache hint to cache results in the result set cache.
  • Improved Plan Caching - plans used by internal materialization and stored procedure plans will be automatically cached in the prepared plan cache. Improvements were also made to reduce the memory footprint of the plans.
  • Refined Load Balancing and Fail Over - clients can use use the statement "SET NEWINSTANCE TRUE" to allow their connection to select a new server instance. See the Client Developer's Guide for more information.
  • Simplified Role Usage - a role can now be assigned to any authenticated user via the any-authenticated attribute on the data-role element.
  • Materialized View Performance - materialized view tables will now automatically create and use non-unique secondary indexes for unique constraints and indexes defined on the view.
  • Binary Web Service Calls - the ws translator now provides an invokeHttp procedure to return the blob contents and string content type of an http/https call.
  • Improved clustering support - see the Admin Guide chapter on clustering.
  • IPv6 support - Teiid can started using IPv6 bind address and can be used with JDBC connection.
  • SESSION_ID - A new system function "SESSION_ID" is added to the system function library.
  • Assignment Syntax Improvements - Teiid's procedure syntax for assignments was clarified so that the assignment value must be a proper expression. INSERT/UPDATE/DELETE update counts must be obtained from VARIABLES.ROWCOUNT, scalar values must be obtained via a scalar subquery. The parser will continue to accept the old syntax and convert the query into the proper form.
  • TEXTAGG - SQL support for Text aggregation. Using this function expression values can be aggregated into a CSV BLOB with configurable delimiter, header, and encoding. The result can be optionally saved to a text file.
See the release notes for more details. Download today, give it try, and let us know what you think and what features you would want in upcoming releases.

The Teiid Team

Friday, October 29, 2010

7.2 CR 1

After a small delay, 7.2 CR1 adds support for converting JSON to XML and creating CSV output via the TEXTAGG aggregate. Numerous fixes have been added from internal testing and excellent user reports.

Please let us know if your resolved JIRAs need any additional work before the final release next week. And now is the time to put in enhancement requests for 7.3, which is still slated for the end of the year.

The Teiid Team

Wednesday, October 20, 2010

7.2 Beta 1

7.2 Beta1 includes support for non-recursive common table expressions (including pushdown support), improvements to multi-source update logic, refined procedure assignment statement syntax, the session_id system function, and various bug fixes.

Give it a try and vote for any issues you want to see in the final release. Time is running out though as we expect a candidate release early next week.

The Teiid Team

Tuesday, September 28, 2010

7.2 Alpha 2

7.2 Alpha2 includes various bug fixes and adds support for replication of result sets and internal materialization in clustered mode. We also added support for IPv6 network addresses.

One major security change in this release is by default Teiid's Admin port is now secure. So if you need to access Admin API from AdminShell or Java classes, before sure to use "mms://" protocol instead of "mm://".

Give it a try and vote for any issues you want to see in the final release.

The Teiid Team

Friday, September 3, 2010

7.2 Alpha 1

7.2 Alpha 1 is now available. We've been working hard to improve plan caching, internal materialized view performance, and simplify role usage. See the release notes for more.

The Teiid Team

Monday, August 30, 2010

Teiid Designer 7.1 Released

Teiid Designer 7.1 is ready for download.

This release further solidifies and enables key data access capabilities for the Teiid 7.1 runtime framework.

The major features of this release are:
  • Restored Preview Data feature
  • Data Role management in VDB Editor
  • LDAP and Salesforce connection profile contributions compatible with Eclipse Data Tools framework
  • Web Services support through generation of deployable web service War files
  • New DDL into Relational Model importer
  • Support for multiple User Defined Function models
  • Tighter integration of Teiid Designer connection management with Eclipse Data Tools

See the release notes for more details.

Interested in getting started with data services? Use our Quick Start Guide and you'll be off and running in no time.

Thank you.

Teiid Designer Team

Wednesday, August 25, 2010

Convert XML data into Relational Table data using Teiid

In this article I will show how to convert XML data into relational form using Teiid using "XMLTABLE" SQL/XML function. XMLTABLE is defined in SQL/XML 2006 specification and is supported by most major databases like Oracle, DB2 etc. XMLTABLE functionality is introduced in Teiid 7.0. If you are already familiar with using this function it is no different in using it in Teiid.

Please see this WIKI article for complete information.

Generating XML data from the Relational Data:

If you need to produce the XML data Teiid provides various SQL/XML functions for that purpose too. Check out this article on the available SQL/XML functions. These are functions are also supported by Teiid.

In a future article, we will how show you how to use these SQL/XML functions and create a web service by just using the Teiid Designer and Teiid.

Tuesday, August 17, 2010

Teiid 7.1 Final Released

Teiid 7.1 continues the 7.x series with numerous features and performance improvements. Here are some highlights:
  • ODBC Support - Support for ODBC access is now available through PostgreSQL ODBC driver clients. See the Client Developer's Guide for more on how to use Teiid's ODBC access.
  • Native Materialized Tables - views marked as materialized without a target table will be materialized internally by Teiid. Materialized view tables will be indexed by the primary key of the view if present and can be controlled through the expanded cache hint. There are also new system procedures refreshMatView and refreshMatViewRow to refresh materialized view table contents.
  • Expanded Cache Hint - The cache hint can now specify pref_mem to indicate that data should prefer to be held in memory, ttl to indicate the time-to-live, and updatable to indicate that a materialized view with a primary key can be updated by row with the refreshMatViewRow system procedure.
  • SQL Support
    • Added NULLS FIST/LAST handling - including pushdown support.
    • Added enhanced numeric stat aggregate functions STDDEV_POP, STDDEV_SAMP, VAR_POP, VAR_SAMP, with pushdown and aggregate decomposition.
    • Added support for the boolean aggregate functions ANY, SOME, EVERY.
    • Added support for using boolean value expression as both criteria and expressions, e.g. "... where boolean_value" or "select x = 1 as boolean_value ...".
    • Changed named procedure syntax to accept param=>value, rather than param=value.
    • Added PRIMARY KEY and the associated index support to temp tables.

  • Parallel Source Queries - reestablished parallel execution of source queries within a query plan along with a prioritized work system to help prevent resource contention.
  • Improved Insert Support - Cross source inserts using a query expression can defer to the source to perform the entire insert atomically with source controlled batching. See maxPreparedInsertBatchSize on any JDBC translator supporting the new InsertWithIterator capability.
  • SHOW Statement - added client handling for the SHOW statement to retrieve query plan information and see parameter values.
  • User Identity - Teiid DataSources in JBossAS connecting to a local Teiid instance can preserve the thread's user identity, rather than requiring a new authentication. See TeiidDataSource.setPassthroughAuthentication.
  • VDB Versioning - The VDB version may be determined by the deployed file name. The expected pattern is .Version.vdb. See the Admin Guide for more on how to use VDB versioning.
See the release notes for more details. Download today, give it try, and let us know what you think and what features you would want in upcoming releases.

The Teiid Team

Thursday, August 12, 2010

7.1 CR2

7.1 CR2 is now available. This should be the last CR prior to the final release. This CR provides an expanded cache hint to control result set cache and materialized view entries. VDB versioning, which allows multiple VDBs with the same name to be deployed at the same time, is now supported through the use of a file naming convention vdbname.version.vdb - dynamic VDBs may use use the vdb.xml version attribute to denote version. See the release notes in the kit and the JIRAs for more information. More documentation changes will be coming before the release.

The Teiid Team

Tuesday, August 10, 2010

Text To Table with Teiid

If you are a programmer, chances are that you have written code at least once to read data from a text file to do something with data. Today, I am going to show you how easy it is convert the text contents into a relational Table data using Teiid with out any code.

If you are ready parse the Text with Teiid, read about it in this article.

In the next blog, I will show you how to convert XML data into a Table, so stay tuned.


Teiid Team.

Tuesday, August 3, 2010

7.1 CR1

7.1 CR1 is now available. This release builds on the alpha release with additional fixes and a revamped temp table system. Temporary tables now support atomic update operations and an optional primary key with a clustered index. This greatly improves the performance of temporary tables for comparison, in, like, and order by operations using the index.

The final release is drawing nearer. Make sure to try out CR1 to ensure your features and fixes are in.

The Teiid Team

Friday, July 23, 2010

7.1 Alpha2

7.1 Alpha2 is now available. The big feature is ODBC access support. From any platform that the Postgres ODBC is available, you can now access Teiid and therefore any source Teiid supports through ODBC.

So if you need ODBC access to a legacy system, then there's no need for a proprietary ODBC/JDBC bridge or loads of custom development for you're own driver - just use Teiid and enjoy all the integration and expanded SQL capabilities as well.

The Teiid Team

Thursday, July 15, 2010

7.1 Pre-release

7.1 Alpha1 is now available. Even though 7.0 was just released, we have been working hard to pack more features and fixes into 7.1. What is already available:

  • SQL Support
    • Added NULLS FIST/LAST handling - including pushdown support.
    • Added enhanced numeric stat aggregate functions STDDEV_POP, STDDEV_SAMP, VAR_POP, VAR_SAMP, with pushdown and aggregate decomposition.
    • Added support for the boolean aggregate functions ANY, SOME, EVERY.
    • Added support for using boolean value expression as both criteria and expressions, e.g. "... where boolean_value" or "select x = 1 as boolean_value ...".
    • Changed named procedure syntax to accept param=>value, rather than param=value.
  • Parallel Source Queries - reestablished parallel execution of source queries within a query plan along with a prioritized work system to help prevent resource contention.
  • SHOW Statement - added client handling for the SHOW statement to retrieve query plan information and see execution property values.

  • ODBC access and many caching improvements will be coming shortly.

    The Teiid Team

    Thursday, July 8, 2010

    Teiid Designer 7.0 Released

    Teiid Designer 7.0 is ready to roll and is now available for download.

    It's been difficult to keep up with all of the important, fundamental changes to Teiid 7.0 but we are finally there.

    This release provides a much more integrated Eclipse experience targeted for developers who want to create, integrate and manage various types of data sources. In Designer 7.0, you will now utilize Eclipse's Data Tools Platform and it's inherent connection profile management framework to assist in both importing/creating relational source models and in testing your VDB's via Teiid JDBC connections.

    Creating and editing your VDB's has been greatly simplified in Designer and this release also provides a quick means to deploy and test your VDB's in a running Teiid server.

    The primary goals of this release were:
    • Provide upgraded tooling for creating and managing deployable Teiid artifacts (VDBs)
    • Simplify VDB creation, editing and management
    • Replace custom connection framework with existing Eclipse DTP functionality
    • Provide seamless source connection transition between DTP, Designer and Teiid
    • Expose basic runtime Teiid API functionality necessary for VDB deployment and data access through DTP
    See the release notes for more details of the work behind this release.

    So use our new Quick Start Guide and go give it a test drive!

    Thank you.

    Teiid Designer Team

    Tuesday, June 29, 2010

    Teiid 7.0-Final Released

    After a long journey (9 months after 6.2 release), we are very excited to announce the release of Teiid 7.0. We revamped the Teiid architecture and removed quite a bit of legacy MetaMatrix to poise our project for future development. At the same time we now integrate seemlessly with other JBoss technologies such JBoss AS, Microcontainer, JOPR, JBossTM, PicketLink, Netty, etc. Teiid is ready to meet your data integration challenges.

    While simplification was a consistent goal of this release, Teiid still supports most of the features available in Teiid 6.2 and earlier. Some notable new features are:
    • The runtime environment. Now Teiid runs inside the JBoss AS. Check out all the benefits in "JCA Container Integration, why Teiid needs it?" series.
    • Simplified Connector development with a split into the Teiid Translator API (largely defined by a single class) and JCA.
    • Enhanced JAAS based security. Many out of the box implementations available.
    • Support for JCA based connectors.
    • JOPR web-based administrative & monitoring tool.
    • Groovy based scripting environment for administrative tasks.
    • Text and web service integration has been redesigned from the ground up to simplify usage.
    • Nearly all of SQL/XML 2003 support has been added, as well as XMLTABLE and XMLQUERY from SQL/XML 2006.
    • The documentation set has been updated and expanded.
    • Many, many and many more features.
    See the release notes for more.

    In our testing, Teiid 7.0 is significantly fastest with improvements to buffering, threading, and caching. So, why wait? download today, and give it a try.

    Teiid 7.0 is just the beginning -- check out Teiid 7.1 roadmap for more exciting features in the pipeline.

    Thank you.

    Teiid Team

    Teiid on Twitter

    Do you use Twitter? Now you can follow Teiid on Twitter

    Now keep track of what is going on with Teiid project and its tools and general data integration articles by following Teiid.

    Wednesday, June 16, 2010

    JCA Container Integration, Why Teiid Needs it? Part 5

    Reason #5 JBoss AS Container

    This reason seems to compliment the title of this article and at the same time contradict some parts of it. Let me explain, it compliments because JBoss AS provides hosting environment of all the reasons described before, and at the same time it takes initial goal of Teiid to be container agnostic. During the development of Teiid 7.0, we tried to be independent of not using any particulars of any single container and sticking to common libraries and specifications, however in few places we had to use JBoss AS specifics like Profile Service, propagating the security contexts etc. Overall, benefits outweigh the hard dependency on JBoss AS. Lets explore the benefits.

    • Provides a JCA environment for Teiid with transactions support. Any connector (resource adaptor) in JBoss AS can be used with Teiid Translator.
    • Single installation. Teiid is installed on top JBoss AS installation, thus, this defines installation directory structure for the Teiid. Since Teiid components are written using the same underlying framework as JBoss AS, they nicely co-exits with other JBoss components.
    • JBoss provides start-up, shutdown scripts. Starting and stopping Teiid no different then starting and stopping JBoss AS server.
    • JBoss provides "logging" framework. Teiid's logging messages are automatically forwarded to JBoss AS log files. No separate log management from Teiid is required.
    • Web Engine. As such Teiid does not require a web environment to operate, however, to support the JOPR based admin tool Teiid requires a web environment to deploy this plugin. At the same time if user's web applications are deployed in JBoss AS, then this provides a single server that avoids unnecessary remote transport between client applications and Teiid.
    • Web services engine. Teiid is an ideal environment for a user to define "data services" for their enterprise. To expose the "data services", Teiid requires web services engine. JBoss AS steps in provides this engine. We have some new improved features and articles coming in Teiid 7.1.
    • Clustering. We have not explored clustering details yet, however Teiid would be able to support a clustered environment. The details will be flushed out in Teiid 7.1
    • Windows service, JBoss AS can be defined as windows service, so Teiid can leverage that feature.
    • Community support.
    There may be other features JBoss AS provides that are useful to Teiid that I am forgetting to mention, however overall this was great integration of two technologies. Teiid team already released 7.0 CR1 previewing these technologies and 7.0 final is coming up soon, so download the Teiid and give it a try for your data integration needs.

    UPDATE: The feature set of Teiid has evolved from time we started writing this series to till now; One major difference is we split the old Connector framework into Translators and Resource Adaptors. This suits better in the JCA environment and does not force fit the Teiid semantics into JCA. This also leads to providing "embeddable" Teiid in future releases.

    Let us know what you think and the features you like to see in upcoming releases of Teiid.

    Thank you.

    Teiid Team.

    Friday, June 11, 2010

    Teiid 7.0 CR1 Released

    We are pleased to announce that after many long hours of work Teiid 7.0 CR1 is available.

    This candidate release refines our separation of the old connector API into light weight Translators and JCA components [1].

    Text [2] and web service [3] integration has been redesigned from the ground up to simplify usage. Nearly all of SQL/XML 2003 [4] support has been added, as well as XMLTABLE [5] and XMLQUERY [6] from SQL/XML 2006 which replace our old xml/relational logic and direct XQuery integration respectively.

    See the examples for a Dynamic vdb that calls and processes REST and RPC webservices from the National Weather Service for getting started with some of the XML features.

    Support for nested table references was added [7], which is a better alternative Teiid's procedural relational style queries. And finally the AdminShell now has both a command line and a graphical mode thanks to a redesign utilizing Groovy [8].

    As you can see there is a lot of new and powerful features available. In details blogs, articles are coming soon. Check it out today -

    The Teiid Team


    Thursday, April 1, 2010

    JCA Container Integration, Why Teiid needs it? Part 4

    Reason # 4 JBoss Microcontainer

    JBoss Microcontainer (MC) is at the core of JBoss AS and provides many essential services needed in developing any enterprise software application. Teiid utilizes MC, through several of its key sub-frameworks, to manage services and configuration/deployment artifacts.

    Service Framework:

    Prior releases of Teiid defined their own service framework(s). These legacy frameworks each had essential flaws - such as not being flexible in defining new services, not providing dependency management, or cumbersome/non-existent configuration management. Teiid in the 6.x releases used "google-guice" for dependency management between services with homegrown configuration management. The service lifecycle was explicitly managed through admin api methods.

    MC provides a POJO based service framework with IOC style dependency injection and full lifecycle management. A simple XML based configuration can your POJO based services along with their state and dependencies. As an added bonus this framework also provides persistence services for the configuration. By moving Teiid to MC, we simplified the definition of our services and greatly improved our dependency management - which eliminated the need for Teiid specific configuration framework. Find more information on MC's component model here. And since MC also supports osgi and google-guice, integration of components developed against other IOC frameworks will work together seemlessly.

    Deployer Framework:

    Teiid requires deployment facilities to handle artifacts such as VDBs, Connectors/RARs, or connector bindings. In prior releases of Teiid deployment and discovery of deployed artifacts was homegrown and based upon different modes of operation - server vs. embedded. This led to inconsistenties in the deployment/discovery code. Dynamic class path management was also led to quite a few issues when JAR resources involved.

    MC has the one of the best deployment frameworks with every flexibility you can imagine. As soon as an artifact is visible at a specific location, the deployer service gets notified. This removes the need to write any scanners for detection of artifacts for your application. This framework gives access to the input stream of the resource(s) that are deployed to be consumed by your service. If you are working with JAR or ZIP based resources, the deployment can be defined to check the structure and validity of the resource.

    The MC framework provides resources through a Virtual File System (VFS) such that you can replace standard file based deployment with any structured repository if you choose to. With the deployer framework you can also construct a class loader based on the artifact deployed. JBoss AS uses this feature to define WAR class loaders. You can read about class loading here. Teiid was re-written to use this deployer framework to load its VDBs (and any contained UDFs). Connectors were re-written as JCA connectors, which are deployed in JBoss AS using RAR and "-ds.xml" files that already have hooks in the deployment framework.

    Management Framework:

    Now that we have defined the services and deployed artifacts, how do you manage them during runtime? Teiid in the previous releases exposed specific administrative interfaces for management of certain resource and did not provide any JMX based management. This led to building Teiid specific Management Consoles that worked only with our administrative functionality. Integration into tools such as BMC patrol, HP Open View etc. for the purposes of management was not possible. For Teiid this was one of the *major* reasons to adopt MC.

    MC provides a framework to define management interface on your beans through annotations. There is no need to define a MBean interface, yet it works like JMX 2.0 and has more features. JBoss also provides away to expose a JMX interface on your beans simply a providing the @JMX annotation - however this still requires you to define an interface. Teiid has not made use of this JMX feature, but will in subsequent releases.

    One thing missing from MC was way to expose the management of your managed beans to external applications. However, JBoss AS luckily comes to rescue by providing the "profile service" through which these beans are exposed. The profile service lets you build your management applications using JOPR based tools. I think development is under way to pull the profile service technology into MC proper in the next release.


    Teiid used MC facilities to re-write its whole management layer. There is simply no comparing the ease with which our services are wired together, configured, and managed with our previous releases. This has created a strong dependency on MC and additional dependencies on JBoss AS. Hopefully this article did a good job highlighting all the benefits of the MC integration. Let's explore the JBoss AS dependency benefits in the next article.

    Tuesday, February 16, 2010

    JCA Container Integration, Why Teiid needs it? Part 3

    The Teiid project is being integrated to run inside a JCA container. In earlier posts I gave two compelling reasons for the move.

    Reason # 1
    Reason #2

    Today we examine #3

    Reason 3: Security

    Security is vitally important for any enterprise application. This is especially true for Teiid as there are typically strict organizational rules governing access to data sources. At a high level Teiid allows for a customizable user authentication/authorization system. There are pre-defined system administrative roles and data authorization roles can be defined for each virtual database (a.k.a. entitlements, or data roles) to govern access at a granular level.

    Teiid 6.2 (and earlier) Security Features

    Teiid 6.2 provided a Membership API to define customizable security domains from which to obtain authentication and authorization information. Teiid shipped with implementations of LDAP and File based membership domains. Security at the connector level was supported, static credentials, client passed credentials, or through "trusted" payloads. Using trusted payloads, the client can pass any object to a connector for custom authentication/authorization. Like previous issues, this worked great! and there are reams of code to prove it. However, there is a better alternative JAAS.

    Java Authentication and Authorization Service (JAAS)

    JAAS is a java based security framework that is built into Java runtime. Here is description from the spec site:

    Underlying the Java SE Platform is a dynamic, extensible security architecture, standards-based and interoperable. Security features — cryptography, authentication and authorization, public key infrastructure, and more — are built in. The Java security model is based on a customizable "sandbox" in which Java software programs can run safely, without potential risk to systems or users.

    JBoss AS uses the PicketLink (JBoss Security) as the security module, which implements the JAAS based authentication framework. Out of the box there are various different login modules available for use. As before, LDAP and File based login modules supported. If their requirements are not satisfied with any of the provided modules, a developer can also write a custom login module.

    By moving into container environment Teiid:
    • replaced a custom security framework with a standards based JAAS based framework
    • has access to a plugin based authorization and authentication mechanism
    • retained all the functionality from before to define security domains.
    • reduced its code footprint.
    Connectors can also be configured for a "security-domain" such that the container ensures the user is authenticated prior to access. In some Containers this security profile is used to create user specific connection pools to segregate connections from common connection pools. Having this login context available at the Connector is similar to having the "trusted payload" as before, however passing a payload is left for the implementation of the login module.

    Next up well look at Microcontiner and its service and deployer framework.

    Tuesday, February 2, 2010

    JCA Container Integration, Why Teiid needs it? Part 2

    Continuing from our earlier discussion "JCA Container Integration, Why Teiid needs it? Part 1", here we will discuss the next reason.

    Reason 2: JCA API

    What is the JCA API?

    From the spec site (J2EE Connector architecture): JCA defines a standard architecture for connecting the J2EE platform to heterogeneous EIS systems. Examples of EIS systems include ERP, mainframe transaction processing, database systems, and legacy applications not written in the Java programming language. By defining a a set of scalable, secure, and transactional mechanisms, the J2EE Connector architecture enables the integration of EISs with application servers and enterprise applications.

    What is a Teiid Connector?

    Teiid connectors also connect to heterogeneous EIS systems, but they implement a Teiid specific Connector API to:
    • Consume commands from the Teiid engine in SQL command object form and translate them into native commands to query and update enterprise sources.
    • Deliver resultant data in a tabular or XML format to the Teiid runtime engine.
    • Provide metadata on what SQL constructs and command forms are supported by the connector.
    How is a Teiid 6.x Connector similar to a JCA Connector?

    In many ways Teiid connectors are similar JCA connectors.
    • The Teiid connector API has many borrowed concepts from JCA API and JDBC API.
    • A Teiid connector can be developed to any EIS to access data.
    • Teiid connectors are packaged in Teiid specific Connector Archive Format (CAF) and deployed into Teiid runtime.
    • Teiid Connector's configuration metadata was defined in special XML file called "configuration.xml" called Connector Type.
    • The connector's runtime properties are defined in another XML file called Connector Bindings.
    • Teiid runtime provides connection pooling for the Connector connections
    • Connector connections can optionally be pooled and participate in XA transactions.
    All this worked well, however JCA connectors provide similar functionality in a standard way. By replacing Teiid Connectors with JCA connectors a lot of duplication in concepts and code are avoided.

    Benefits of the JCA API with Teiid
    • Provides standards based feature rich API for developing connectors. Teiid only needed to define extensions to this API to provide Teiid specific functionality.
    • Standard packaging structure. JCA provides a RAR file format, which can embed all the resources specific to this connector in a single artifact. This replaces the non-standard Teiid CAF file.
    • Defines a class loading scheme for RAR files that solves common class loading issues involved in Teiid connector deployment. The old CAF file did not define a classloading scheme, but relied on "extension" modules.
    • Provides a standard way to define the connector's configuration medata in a "ra.xml" file. This eliminates the need for defining connector types in Teiid.
    • Availability of various JCA containers like JBoss AS to deploy your connectors.
    • JCA provides support for participating in Local and XA transactions.
    • Built in support for various security concerns.
    • Developer familiarity.
    Previously functionality like configuration, depolyment, runtime management, connection pooling and transactions support was provided by the Teiid runtime. Now as a JCA connector, the JCA container (like JBoss AS or even GlassFish) provides a similar feature set.

    Benefits of a JCA Container
    • A standard deployment model across containers. Containers know how load these connectors into separate class loaders defined by the J2EE spec. There is no need for "extension" modules in Teiid runtime to load dependent JAR files into system.
    • Built in connection pooling by container. Teiid's connection pool, while optimized for performance, is not as feature full as those offered by a container.
    • Container defined deployment model. For example JBoss AS defines a xml file format "-ds.xml" to configure runtime properties of this connector. Previously Teiid defined it's own XML format to define the connector binding.
    • Access to container management tools to configure, deploy and manage connectors. Teiid specific management is not required.
    • Access to a wide variety of JCA connectors for different EIS sources by different software vendors. These can be easily extended to use with Teiid thus increasing Teiid's ability to integrate more sources.
    • Variety of JCA containers to choose from, like JBossAS, GlassFish, WebSphere, etc.
    • Developer familiarity. If you know how configure a data source under container, you will understand how to configure a Teiid connector.

    The Teiid Engine with JCA

    The Teiid engine itself is also being developed as a JCA connector. While the engine does not directly connect to EIS, it still needs access to standard functionality provided by the JCA API.

    XA Transactions:

    The Teiid 6.x runtime provided transaction support using an embedded instance of JBoss TM. Even though Teiid used an external transaction manager, coordinating XA transactions is a complicated task especially in a multi-threaded multi-source environment like Teiid.

    By using a JCA container as the runtime environment these tasks are delegated to the container, and since the JCA API provides access to the "XATermintor" interface, Teiid can still control the start/commit/rollback of transactions initiated through our JDBC client. Now the Teiid runtime does not explicitly depend on a particular transaction manager.

    Multi-Threaded Environment:

    The Teiid runtime is a multi-threaded environment with specialized queues to ensure highly parallel processing. Our processing model was also easily adapted to the JCA API, since it provides access to the "WorkManager" interface that fronts a container managed thread pool.

    The article shows that the JCA standard was the gateway to integrating Teiid into a container, but it is only a start. We'll cover security in the next article, stay tuned.

    Tuesday, January 19, 2010

    Teiid Designer 6.3 Available

    Due to Eclipse packaging and project version dependencies, Teiid Designer 6.2 was not compatible with Eclipse 3.5. This required updating to newer versions of Eclipse EMF and UML2 projects which resulted in some minor code changes.

    So this 6.3 release keeps Teiid Designer in sync with Eclipse.

    Note that the embedded version of Teiid remained the same within Designer, namely Teiid 6.2.

    Visit our download page for more details.

    Next our team will focus on a Teiid Designer 7.0 release which will provide tooling changes required to keep pace with Teiid 7.0 release.

    Barry LaFond
    Teiid Designer Project

    Tuesday, January 5, 2010

    JCA Container Integration, Why Teiid needs it? - Part 1

    A huge architectural change for the Teiid runtime is upcoming up in Teiid 7.0 release. Teiid is being integrated into a JCA container - JBossAS 5.1.0 GA. In this new platform you can expect all the data integration features of Teiid with power and flexibility of JBossAS.

    If you are thinking, Teiid is a relational data integration engine what does it has to do with a JEE container? This has little do with Web technologies. It is all about JCA (Java Connector Architecture) and the advanced development features of JBossAS. Let me explain each reason in a multi-part series of articles.

    Reason 1: Right ingredients. Integrate with modules designed by the domain experts.

    Even though Teiid is a data integration engine, the inner runtime workings of Teiid apart from the command parsing, relational planner and optimization engine are much more similar to any enterprise server application. Like security, user management, resource management, log management, caching, availability, scalability, reliability. How about the usage of current day's POJO based IOC containers, Management APIs and various tools associated with them? Over the past 7~8 years of MetaMatrix/Teiid development all of these features were home grown per customer need. However many of these features were not developed per industry standards since no relevant standards existed at the time nor did customers demand standards compliance. As a result Teiid has ended up with tens and thousands of lines to code to replicate what has become common container functionality. The Teiid Team claims to be experts in the data integration arena and leveraging an alternative runtime platform allows us to maintain a sharper focus.

    Not coincidentally the JBoss AS 5 platform, built using the highly extensible and feature rich Microcontainer environment/framework, provides not only a JCA container, but a POJO based service framework with dependency injection, AOP, virtual file system, deployer and management frameworks which are ideal for developing robust software. JBoss AS is also a vibrant and exciting community ready to help with any issues. So, for Teiid we choose Microcontainer as the underlying framework to redesign Teiid's service components and JBoss AS host environment.

    This will not only dramatically reduce the code foot print of Teiid, but will leverage a wealth of existing documentation and knowledge around utilizing JBoss AS, Microcontainer, and JCA. We still however did not talk in-depth about the JCA container part of it - that will be coming up in the next article.