Wednesday, November 25, 2009

Teiid 7.0 MS 1

Just in time for Thanksgiving we're proud to announce Teiid 7.0 M1 is available. This milestone of 7.0 adds support for order by expressions, updatable temp tables, arbitrary quoted identifiers, and h2/hsql translators. Also JDBC DatabaseMetaData and the system tables have been revamped, and there's a host of other fixes.

Before rushing off for the download, you should know Teiid 7.0 does not yet have a corresponding Teiid Designer release - it should be available within two weeks. You can of course still use Teiid 7.0 through the dynamic VDB feature. Existing VDBs from older Designer releases may still work with 7.0 - see TEIID-145 and TEIID-871 - for compatibility issues. Also while milestone 1 uses the same directory structure/installation procedure as Teiid 6.2, subsequent milestones will dramatically change the deployment procedure for Teiid.

Be sure to vote/comment on remaining issues you want to see addressed in the 7.0 release.

Tuesday, September 22, 2009

Teiid 6.2 Released, packed with new features

The Teiid project team is excited to announce the release of version 6.2.0 - available for download here. The 6.2 release adds significant features in ease of use and enterprise readiness. It's never been easier to integrate your disparate enterprise data sources.

Dynamic VDB


An often requested feature, with a dynamic VDB data can be integrated between multiple sources very quickly without explicitly constructing a VDB using Teiid Designer. Check out a new example (dynamicvdb-portfolio) in the Teiid download, and read this blog for more info.

Introducing Teiid Server

In the prior releases of Teiid only an embedded deployment mode was offered. This meant the Teiid instance could not be shared among multiple clients. Now with Teiid server, you can run a standalone Teiid process and connect remotely over a highly scalable and configurable socket transport. Look here for installation instructions and how to connect to a server.

Security

Multiple facets of security were added to help safe-guard your data.
  1. User Authentication: when enabled, users must be authenticated before they can access Teiid resources. You can configure a Teiid membership domain to plug into LDAP for authentication or use a simple file based authentication for development purposes. Visit identity support for more information.
  2. User Authorizations: access to Teiid can be controlled through specific grants of authorizations. These fall into two categories. "admin-roles" manage access to Teiid configuration and deployment. "data-roles" manage access to data exposed by a VDB. The latter is also called as "data entitlements", and can restrict data access up-to a "column" level. Visit managing authorizations for more information.
  3. SSL: when connecting to Teiid over the socket transport you can use SSL configured with your own certificates. See this.
  4. Encrypted Passwords: when defining data sources often the passwords are left in the clear text in the configuration files, Teiid provides a solution for encrypting them.
Logging

Logging has been bought under the management of Log4J. There are new logging contexts that allow you to capture SQL "commands" flowing through Teiid system and capture "audit" messages related to security and admin events. Learn more about how to manage logs here.

Admin Shell


Admin Shell is script based management tool for Teiid. Users can run Admin Shell in an interactive command line mode or script mode to connect, query and administer Teiid. Check out the Admin Shell Guide on documentation area and grab the binary in download area.

Many bugs, query constructs, and planning enhancements are also included in 6.2. Here is the complete list of changes. Take it for a test drive and let us now what you think.

Thank you

The Teiid Team.

Tuesday, September 8, 2009

Teiid 6.2 M3 and JBoss World slides

Teiid 6.2 M3 is now available. It fixes several issues from M2 and cleans up quite a bit of logic in the JDBC Connector. Next up RC1.

Have a look our presentation from JBoss World. It provides a good introduction to our internals - architecture, planning, processing, etc. If you have any questions feel free to post to the users list.

The Teiid Team

Wednesday, August 26, 2009

Data Integration in 5 minutes

Data integration between multiple disparate sources is now as simple as writing SQL code in your application as if all your schema (tables, views and procedures) from different databases are co-located in a single data source.

No, I am not talking horizontal partitioning of your data that have same schema defined in all the databases that some other tools support. You can have segments of data to your liking or the way they were given to you. That not all, you can even add text files to this mix and use them as another source that is participating in data integration. All this is doable with Teiid.

All you have to do is to define the "data source(s)" you would like to integrate in a Teiid configuration file and configure Teiid as your JDBC data source in your application and start writing your application, which may be a simple JDBC or Hibernate application.

Sounds simple huh? Thanks to a new feature called "Connector Supplied Metadata", which is being introduced in the Teiid 6.2 release. This feature is so powerful that using it you can write a single EJB that reads or writes data to/from multiple sources, or you can write Hibernate application with single "Session Factory" that has domain objects from multiple different sources, where you can write Hibernate HQL query that joins two tables from different sources.

So What? Teiid had this capability all along!

Yes, you are right. There is nothing new in terms of additional capabilities of Teiid, it always supported data integration as its core capability. What is new is how it does it, and how flexible it is, and how fast you get from problem to the solution. Prior to Teiid 6.2 release in order to integrate two or more sources in Teiid the following steps must have to be taken.




Step 1: Using Teiid Designer, you need to import and create Source models

Step 2: Using the Teiid Designer, you need to create a VDB and add model files to VDB and configure the "Connectors" for the sources.

Step 3: Deploy into Teiid runtime.

Now, contrast the above scenario to below

as you can see the complete "modeling" step and building a VDB is eliminated (Step 1&2 from above) and metadata that is required for data integration is imported dynamically at runtime and working models are built automatically using "Connector Supplied Metadata" feature. However, user still needs to supply a configuration file where they can specify information about their sources! This configuration file called "Dynamic VDB". And you are required to deploy this configuration file into Teiid runtime engine.

This model does have its pros and cons, and may not suit all the data integration needs.

Pros
  • Metadata about your source imported dynamically at runtime, you have an option to cache after the first import.
  • No static model of your metadata, so it can keep changing as you change the source structure.
  • No Teiid Designer tool to use, cutting down on time and learning curve with Designer.
  • No VDB, i.e no custom artifacts to define your metadata, just define a simple XML file with your favourite editor.
  • Possible to write dynamic discovery of sources and integrate them on the fly.

Cons:
  • You can not define virtual tables, procedures composing of multiple sources.
  • You can not define a single view of your data.
  • Many enterprise features may not be available.
  • Currently only JBDC and Text based sources are supported, but more coming soon.
Teiid still supports both the Designer based VDB and Dynamic VDB modes. If you need a simple and quick data integration use the "Dynamic VDB", otherwise use the Teiid Designer to create a virtual database as before, after all it is all about flexibility.

So, next time you think about combining data from your Excel sheet with your Oracle data, do not custom code the integration layer, use Teiid instead. Give it a try! and let us know what you think. Thank you for reading.

You can check out a quick example here.

Thanks.

Ramesh..

Monday, August 17, 2009

Announcing the Teiid Mid-America Media Blitz

There are upcoming Teiid presentations at JBossWorld in Chicago on Friday, September 4 @ 9:45 and at The Strange Loop in St. Louis on Thursday, October 22 @ 11:40. These will go into more detail about the engine internals than the introductory booth demo from the last post. If you can attend either session, feel free to stop by with any questions - otherwise stay tuned and we'll post the slides.

The Teiid Team

Teiid Booth Demo from JavaOne

We did a quick demo of Teiid and Teiid Designer in the JBoss booth at JavaOne. The demo is an expanded look at our quick start and highlights our ease of use as a JDBC source in Eclipse Data Tools and Hibernate. Check out the videos here:

Part1: http://www.youtube.com/watch?v=Fejt8LsmOtI

Part2: http://www.youtube.com/watch?v=9fpI3zwfd5Y

Part3: http://www.youtube.com/watch?v=aF8G3ZKLuAc

The Teiid Team

Friday, July 17, 2009

Teiid 6.2 Milestone 1

The 6.2 is shaping up to be a feature packed release, and M1 is now available to try them out. While still supporting embedded mode as before, the Teiid kit can also be run in server mode with support for SSL connections. Either mode now supports encrypted passwords, pluggable authentication/authorization, and greater configurability of logging. The kit also includes an example integrating the portfolio example without the Designer. See http://www.jboss.org/community/wiki/TeiidConnectorSuppliedMetadata for more of direct usage of the JDBC and Text connectors.

Stay tuned for the next milestone with more features, documentation, and of course fixes.

The Teiid Team

Thursday, June 11, 2009

Teiid 6.1.0 Released, adds Hibernate Dialect for Teiid

We are excited to announce the availability of Teiid 6.1.0 along with a coordinated release of Teiid Designer 6.1.0.

There are numerous fixes and improvements in this release. See the complete list here.

Some of the highlights of the release include:
  • Simplified embedded deployment and configuration. Deployment of embedded in JBoss AS 5.0, Eclipse DTP, Squirrel, etc. is straight-forward. See some examples on our wiki.
  • Resolved class loading issues surrounding the usage of vendor specific drivers.
  • A "Hibernate Dialect" for Teiid is available. Accessing the your "virtualized" data is now as easy as creating a VDB and writing a Hibernate application.
  • Enhanced processing with the introduction of a partitioned merge join, sort optimizations, and a variety of new push down features.
  • Stream based processing for XML connector.
Enjoy the features, please be sure let us know what you think. We always welcome your comments and suggestions!

Thank you.

Teiid Team.

Thursday, June 4, 2009

A Relational Data Integration Engine

Teiid has been an open source project for few months now and the response from community has been fairly positive. New users are taking it for test drive and we have busily been executing on our roadmap. Still I sometimes feel, we may have not explained what Teiid software really does?

Since I am java six pack, I am going to dumb it down to level that I understand, and present it to you.

If you look around Teiid site depending, you will find a multitude of terms and definitions, here are some examples

  • It is a EII (Enterprise Information Integration) tool

  • It is a Virtual Database

  • It is Data Integration Engine

  • Data Mash up Engine

  • Data Virtualization Engine

All the above are accurate. But, I want to describe Teiid's base processor more precisely.

"Teiid is a Relational Data Integration Engine"

Using Teiid and its supplied tools, you build a brand new logical data model for your data in multiple heterogeneous data sources using SQL as glue language. Now, the schema from this logical model is what you can expose to your end applications as domain model.

So, in effect you took multiple existing data sources, and defined relationships among the data you have, to create a data model you need without much effort, i.e. you got model driven data integration.

Once this logical schema is deployed into a Teiid runtime engine, Teiid behaves like a relational database. It provides a secure, preferment and transactionally aware system that can be accessed by your end user applications. Note, that the physical location of your data still be in the same place as before, however the logical model results are built at runtime for you by Teiid.

Well, all is good so far but I said, "Teiid is relational data integration engine" does this mean Teiid can only integrate data from multiple relational databases? absolutely not! Teiid can integrate data from any RDBMS like Oracle, SQL Server, DB2 etc. as well from Text files, Web Services, Excel sheets, XML, Main Frame or any of your custom data. If you are wondering how that is possible? The answer lies in Teiid Connectors.

Teiid Connector: A connector is data access mechanism to a "source" system like DBMS, file or a web service. Also, another very important function of the Connector is that it maps the underlying source's data model into relational entities (like tables and procedures etc). So, naturally when you are dealing with RDBMS sources they represent the sources underlaying data model. However, if a connector represents a CSV file, it exposes the metadata information about csv data in "relational" terms to the out bound systems, thus a CSV file looks and behaves like a RDBMS source to a calling system. Using this technique can can virtually transform any source and represent it in relataional terms. Thus you can even call Teiid as "Relational Data Abstraction" engine. So, if you have non standard source of data you can map that into relational terms and use it as RDBMS. Now, as you can imagine, magically all the sources under Teiid system look and behave like relational sources. Now on to integration of these sources.


Query Engine: Now let's assume, a Teiid system is setup with two separate connectors. When a end user application submits a SQL based query, the Query Engine uses the metadata/schema of the logical model to parse, validate, optimize and split the query into multiple source specific queries. The query engine will parallelize the source access if possible and process the data into the final result. The query engine has numerous optimizations. The most important one is that it pushes the maximum amount of work to the source connector, thus avoiding any expensive in-memory operations.

End user applications can access the Teiid mainly using JDBC API. That means you could use Hibernate easily and never see any difference. With some additional tooling you can even access your data as Web Service. Teiid project provides tools for every aspect of the development and deployment of model driven data integration.


Before I end this article, there is one very important concept I want leave with and that is Teiid is not a ETL or Data Mart tool, where the data is duplicated from the original source then served to target systems. Teiid works with data from their original locations as it is, there is no unnecessary duplication/coping of data.


Now that hopefully you know "What Teiid really is", you can see how it can solve your company's data integration needs easily. Let us know how we can help.


Ramesh..

Thursday, May 14, 2009

Teiid Designer 6.0.0 Released

We are pleased to announce that Teiid Designer 6.0.0 is now availailable.

Teiid Designer is a visual tool that enables rapid, model-driven definition, integration, management and testing of data services without programming using the Teiid runtime framework.

Just click Download now to get started and visit our project page at: www.jboss.org/teiiddesigner for more details.

Does your data reside in different databases or systems? Got DB2, Oracle and SQL Server? Doesn't matter. Teiid Designer can import your schema and create reusable, queriable relational models. So install Designer and try out its data virtualization modeling framework and import from multiple, related or disparate data sources. Then transform your data into virtual views or tables that better represent the needs of your business applications.

Take it further and create a Virtual Database of your models in Designer and execute queries against the VDB deployed in Teiid embedded within Designer to exercise the power of federated data access.

Note that this release is functionally compatible with Teiid 6.0.1.

Barry LaFond
Teiid Designer Project Lead

Tuesday, April 28, 2009

Teiid Designer Progressing

We've made substantial progress toward a full release of Teiid Designer 6.0.0. Taking a half million lines of Java code into open source has been a daunting task and we're almost there.

To hold you over until 6.0.0 is ready, we released a second SNAPSHOT for download. This SNAPSHOT provides full functionality of Designer's Preview Data feature as well as delivering native JDBC driver templates to assist in quickly getting you set up to import your source schema for model creation. All you'll need to provide are the driver jar's themselves.

Note the following features are not fully functional due to Extension metamodel problems incurred as a result of our EMF upgrade from version 2.1 to 2.4.
  • Extension model creation and editing
  • Web Service as Relational Source Model importer
  • XML Schema as Relational Source Model importer
  • Salesforce as Relational Source Model importer
So install this latest pre-release version and try out Designer's data virtualization modeling framework. Import from multiple, related or disparate data sources. Does your data reside in different databases or systems? Got DB2, Oracle and SQL Server? Doesn't matter. Teiid Designer can import your schema and create reusable, queriable relational models. Transform your data into virtual views or tables that better represent your business needs.

Want to go further? Create a Virtual Database of your models in Designer and execute queries against the VDB deployed in Teiid embedded within Designer to exercise the power of federated data access.

... oh... and give us your feedback!

Barry

Monday, April 27, 2009

Teiid Update..

Today Teiid Designer team released a snapshot relase of upcoming 6.0.0 release kit. This version of Designer works with the Teiid 6.0.0 release. We were able to release the Teiid embedded runtime ahead of the time, however this snapshot release is significant as this provides much needed tooling support for the Teiid 6.0.0.

If you are using Teiid pre-6.0.0, you would need to download both Designer and Embedded kits. Apart from fixing many issues, the release introduces the support for native vendor driver support for Oracle, DB2, SQL Server. Teiid previously already supported MySQL, Derby, PostGres along with Data Direct specific JDBC drivers and various XML and Text sources. We will be adding more supported drivers in future releases, however, if you are looking for support for a specific data source and one is not available, please let us know.

In this release, you would be able to import metadata, and preview data and build a connector in one easy step. Thanks and kudos for new classpath editor and workflow changes by elivisisking which removed hassle of manually setting the classpath for connectors, which makes creating Connector automatic in a snap!

Please note to support this snapshot release we have re-kitted the Teiid 6.0.0 embedded kit with some necessary patches, so if in case you have downloaded the Teiid 6.0.0 before, please download it again for the updated one.

Thank you.

Ramesh..

Wednesday, February 11, 2009

What the heck is Teiid?

If you wondering what the heck is Teiid, and what kind of crazy name is this, well Teiid is data lizard, actually it is a "Whip Tail Lizard" and very interesting one too.

It is pronounced as "tee-id". Here is definition of it by http://www.thefreedictionary.com

The Common Data Lizard (Cnemidophorus datumus) is the only species of whiptail lizard native to all continents. It was once considered a subspecies of the Common DeeBase Whiptail, Cnemidophorus Jdbcus, but was eventually granted full species status due to it's uncanny reliability and availability. It is one of only lizard species known to be embeddogenic.

Description

The Common Data Lizard grows to between 8 and 12 inches in length. It is typically blue-gray in color, with 5-6 white or orange dots decorating the hind section of the of the body. They are thin bodied, with a long tail.

Behavior

Like most whiptail lizards, the Common Data Lizard is active, yet stable. They are acute, energetic, and fast moving, and fear nothing when approached. Its preferred habitat is clean, arid areas with ample storage space.


Here is wikipedia notes on teiidae

Images & Videos

Here is are some images of Teiid from google

http://animal.discovery.com/tv/a-list/creature-countdowns/lovers/lovers-09.html

Here's a teiid in action!

Auditioning for Chiller Theater...


Why this name is special?

We used this cool site to come up the name (thanks), check it out.

Well, this is one of couple names that embeds "EII (Enterprise information Integration)" in it, the space in which this project plays major role, and considered to be THE best and complete open source solution available in the community. More over he does make a good mascot.

We have just released Teiid 6.0.0, which is our first community open source release. So, check out Quick Start Example, and see how this works.

Try it out, and share your comments and suggestions.

Thank you

Teiid Team

Tuesday, January 20, 2009

Teiid invades the community.

We are very proud to announce a new open source project, "Teiid" under JBoss.org's community of projects. Teiid is a data virtualization system that allows applications to use data from multiple, heterogeneous data stores and provides you with single view of that data. Teiid (pronounced Tee - id) is also a whiptail lizard.

Although the Teiid project is new to open source, the Teiid technology has been around for many years under the "MetaMatrix" brand name. Red Hat acquired "MetaMatrix" in April 2007 and committed to open sourcing the technology and Teiid is result of that effort.

During this transition time, Teiid has gone though several major updates in its technology
  • SVN migration
  • Projects consolidation, refactoring into manageable components.
  • Use Maven for build system
  • Replacement of transaction framework to "JBoss Transactions"
  • Use of JBoss Cache for caching and clustering needs.
  • Upgrade to Java 1.6
  • Re-write of Query Processing Engine
  • Major updates to Query Optimizer code.
  • New project web site
  • New issue tracking system
  • Teiid Designer upgraded to Eclipse 3.4.1
  • Teiid Designer is deployed as Eclipse plug-in

and many more smaller features. For more information on Teiid project please visit http://www.teiid.org or http://www.jboss.org/projects/teiid

Many more new features are planned in our roadmap, so give it try and let us know how Teiid can satisfy your enterprise data virtualization needs.

Thank you.

Teiid Team.