Friday, December 19, 2014

Teiid 8.10 Alpha2

Teiid 8.10 Alpha2 is now available from the downloads and maven.  Feature highlights include:
  • TEIID-991 SybaseIQ Translator - for use with SybaseIQ 15.1 and later.
  • TEIID-2882 PrestoDB Translator - for use with PrestoDB 0.85 and later.
  • TEIID-1091 Additional SQL/XML Support - XMLEXISTS, XMLTEXT, and XMLCAST were added.
  • TEIID-3216 Update Support for Google Spreadsheets
Ramesh also introduced logic to support OAuth - Integrating Twitter With OAuth.

There have been several JIRAs to increase compatibility including quoted set/show variables, temp table on commit clause, (the long overdue) -- single line comment, and set session characteristics for transaction isolation level.

And of course there have been a host of other issues resolved - many of which originated from community users.  Special thanks to Filip Elias for contributing the update support for google spreadsheets.

Steve

Thursday, December 18, 2014

Expose Twitter data in Teiid, using OAuth Authorization

If you want to access Twitter data or any other REST web service (google, facebook etc) secured by OAuth 1.0A or OAuth 2.0 authentication schemes using Teiid, please read this article https://developer.jboss.org/docs/DOC-52995 for an example.

This feature will be available in Teiid starting from 8.10.Alpha2 release.

Thanks.

Ramesh..

Teiid Designer 9.0 Released

Teiid Designer 9.0 Final is now available via our update site or downloadable archive.

This release focused on improving existing features and bug fixes.

Primary drivers for this release were:
  • Upgrade to Eclipse 4.4 Luna target platform and JBDS 8.0 GA
  • Compatibility with Teiid 8.8 Runtime
  • Bug fixing with special attention to issues on Windows OS

Check out our What's New for 9.0 article for details...

Barry LaFond
Teiid Designer Team

Tuesday, December 2, 2014

Teiid 8.10 Alpha1

We've made steady progress on 8.10 and Alpha1 is now available from the downloads and maven.  Feature highlights include:
  • TEIID-3192 Salesforce Connectivity - connectivity was converted to the partner api provided by Salesforce rather than utilizing CXF. Please log an issue if you were utilizing features from the CXF config.
  • TEIID-3046 Recursive Common Table Expression - engine and push down support was added.
  • TEIID-3163 TEIID-3167 Text Table Enhancements - to support a general no trim option and to use a different row delimiter.
We expect progress with Olingo, but it appears that their final won't be available until March.  Due to that and other needs we will likely produce an 8.11 as well.

There will be several new sources coming in the 8.10 release including SybaseIQ and PrestoDB.  HBase is likely and there should be a couple more if time permits.

Thanks in advance for the community effort as we work toward the 8.10 release and be sure to let us know if there are issues needing attention.

Steve

Monday, November 17, 2014

Teiid 8.9 Final Released

The Teiid community is excited to announce yet another (nearly) time boxed release of its Data Virtualization software - version 8.9. This release took an extra 4 weeks on top of regular 12 weeks to accommodate a NEW version of the host environment - JBoss EAP 6.3.0 Alpha.

You can find the Teiid downloads at http://teiid.jboss.org/downloads/

A little over 100 issues were resolved and this is looking to be solid release. The major features include:
  • TEIID-3009 WITH project minimization - common table expressions will have their project columns minimized.
  • TEIID-3038 geoSpatial support for MongoDB translator
  • TEIID-3050 Increased Insert Performance with sources that support batching or insert with iterator.
  • TEIID-3044 Function Metadata is available through system tables and DatabaseMetaData.
  • TEIID-1910 TeiidPlatform for EclipseLink integration is now provided via the teiid-eclipselink-platform jar in maven.
  • TEIID-3119 Performance improvements in grouping and duplicate removal as well as general improvements to memory management.
  • TEIID-3156 Collation aware prevention of order by pushdown via the collationLocale translator property and the org.teiid.requireTeiidCollation system property.
  • TEIID-3042 Usage Information on views and procedures in the system table SYSADMIN.Usage.
  • OData4 Support - There is partial support for OData4 using the Apache Olingo project. The OData2 is still intact. We consider this still as experimental feature.
Thank you for continued support in the community, especially to the individuals who found issues, entered JIRAs, and provided other valuable contributions. Huge thanks to Alex K.,  Bram GadeyneCristiano NicolaiDevesh MishraGary GregoryHaifen BiHarrison GentryIvan ChanJoao ViragineJoseph CHIDIACMark AckertMark AddlemanMichael FarwellPranav KSalvatore RSanjeev GourSunil VarmaTom Arnold for all your contributions.

The development of Teiid 8.10 is already well underway, but there is plenty of time to add more features. The target release date will be February 15th, 2015. If you have been thinking about a feature for Teiid, this is a great time to engage the leads and submit your request/proposal. We really need experts out there to help with our big data story.  If you have experience with NoSQL stores, please write a translator in Teiid or improve what we already have in the build.  Check out the Open To Community issue bucket to see some existing issues that could be good to start with.  We are also looking for web smiths, as we really want update the main Teiid website - any volunteers are welcome.

If you are thinking about production support, look into Redhat Data Virtualization.  You can also download a free developer version of upcoming DV 6.1.0 version there.

Thank you. 

Steve and Ramesh.

Thursday, November 6, 2014

Teiid 8.9 CR3

It seems that CR2 was well received or at least there were no major issues with the switch to EAP 6.3 Alpha.  So now we are moving onto 8.9 CR3, which is available in the downloads and maven.  The final release should be expected within the next 10 days.

There are a couple of fixes beyond what was in CR2 and we are up to 116 total issues addressed for this release. 

Work has begun on 8.10.  Expect an 8.10 Alpha about a week after the 8.9 Final.  Now is a great time to vote for or log issues to be included in that release.

Thanks,
Steve

Tuesday, October 28, 2014

Teiid 8.9 CR2

We are happy and trepidatious about the availability of 8.9 CR2 in the downloads and maven.  This moves us closer to the final, but introduces two important and late changes:

  • The target platform is now EAP 6.3 Alpha.  This platform change better aligns us with more recent product releases and removed the need for applying the RESTEasy patch. We also have a high degree of confidence in the changes as we have already been through this effort for productizing DV 6.1. You may however need to be signed into a jboss.org account for the EAP 6.3 Alpha download - we are working with the EAP team to get that restriction removed.   
  • The semantics of the pg transport security configuration have changed. By default the socket configuration has SSL disabled.  If login security is used, then clients will be required to use a secure login - which is currently only GSS.  If SSL is fully enabled, then clients will be required to use SSL. This more closely matches the behavior of the JDBC transport.  There is a backwards compatibility option if needed - see System Properties.
Given these changes we'll expect another CR release before the final.  We apologize for the delay as we typically do want to stick as close as possible to a quarterly release.  

Thanks again for all of the community effort in logging issues.  We are up to 110 addressed for this release, so we are expecting a high quality result.  Work will begin shortly on 8.10, which means that now is the time to vote for or log issues to be included in that release.

Steve

Sunday, October 26, 2014

The 3 big problems with data and how to avoid them


The 3 big problems with data and how to avoid them

Register Now

You’re invited to attend the Beyond Big Data Webinar Series, a set of 5 Red Hat webinars. Read more about the entire series.

Webinar 1: The 3 big problems with data—and how to avoid them
November 5, 2014, 11:00 a.m. - 12:00 p.m.

No matter what your organization looks like, chances are you're wrestling with at least one of the following data challenges:
  • Data silos that are difficult to access when needed
  • Point-to-point integration that doesn't scale
  • Data sprawl leading to security and compliance risks
Join this webinar to learn how to implement a data strategy and architecture to avoid these problems.

Speakers:
Syed Rasheed, solution marketing manager, Red Hat
Syed Rasheed, solution marketing manager at Red Hat, coordinates marketing, evangelism, and consulting activities. In addition to helping customers address integration challenges, he is responsible for working with customers, partners, and industry analysts to ensure next-generation Red Hat technology meets customer requirements for building business process automation and integration solutions. Syed is an 18-year veteran of the IT industry with extensive experience in business process management systems, business intelligence, and data management technologies. His work spans several industries, including financial services, banking, and telecommunications.

Ken Johnson, director of product management, Red Hat
Ken Johnson, Red Hat director of product management, is responsible for SOA and data integration products and technologies. Prior to joining Red Hat, Ken was a senior engineering manager at MetaMatrix, Inc., a pioneer in the enterprise information integration (EII) market. He has also held technical leadership positions at Vignette Corporation, Oberon Software, and Sybase, Inc., with a focus on application integration and data management technologies.

Register Now

Wednesday, October 22, 2014

Single sign-on (SSO) with Teiid with Kerberos secured Web Service

With the latest upcoming release of Teiid 8.9, user can achieve single sign-on (SSO) in accessing any REST or SOAP based web services that are secured through kerberos.

In the previous installment, I have shown few articles on how to do Kerberos authentication through Teiid/JBoss EAP in consuming a REST web service.

In this blog I will showcase few more articles that are designed around SOAP web services. Mostly these articles are written tutorial fashion such that one can easily follow and create and as well as consume the web services in JBoss EAP and Teiid.

1) How to implement a SOAP Web Service with Kerberos authentication in JBoss EAP

2) How to implement Kerberos authentication to a SOAP Web Service using Teiid


In article (1) shows how to create sample SOAP web-service that can be secured through Kerberos

In article (2) shows how to consume web-service in Teiid that is secured through Kerberos

In article (3) show a SSO scenario, where Teiid is secured through Kerberos, and how the authenticated token can be used  to access a same web service that is secured also through Kerberos.

Next step we want to provide consuming services that are secured through SAML & OAuth. If you are familiar with implementation of these technologies in CXF and can help implement a solution, please let us know.

Thanks.

Ramesh..

Friday, October 17, 2014

Teiid Platform Sizing Guidelines and Limitations

Users/customers always ask us about the sizing of their Data Virtaulization infrastructure based on Teiid or the JDV product from Redhat. Typically this is very involved question and not a very easy one answer in plain terms. This is due to fact that it involves taking into consideration questions like:
  • What kind of sources that user is working with? Relational, file, CRM, NoSQL etc.
  • How many sources they are trying to integrate? 10, 20, 100?
  • What are the volumes of data they are working with? 10K, 100K, 1M+?
  • What are the query latency times from the sources? 
  • How you are using Teiid to implement the data integration/virtualization solution. What kind of queries that user is executing? Even small federated results may take a lot of server side processing - especially if the plan needs tweaking.
  • Is materializing being used?
  • Is query written in optimal way?
  • and so on..
Each and every one of the question affects the performance profoundly, and if you got mixture of those then it become that much more harder to give a specific configuration.

Before you start to thinking about beefing up your DV infrastructure, the first thing you want to check is:
  • Is my current infrastructure serving my current needs and future expectations?
  • What kind changes your are expecting?
  • Is there a change in type of sources  coming, like using Hadoop or using cloud based solutions?
We need to build the DV infrastructure on based on these available resources combined with mandated requirements for your usecase. Since Teiid is real time data virtualization engine, it heavily depends upon the underlying sources for data retrieval (there are caching strategies to minimize this). If Teiid is working with slow data sources, no matter much hardware you throw at it, you still going to get a slower response.  The place where the more memory and faster hardware can help DV is, when Teiid engine doing lots of aggregations, filtering, grouping and sorting as result of a user query over large sets of rows of results. That means all the above questions I raised may directly impact based on each individual query in terms of CPU and memory.

There are some limitations that Teiid engine itself has:

1.  hard limits which breaks down along several lines in terms of # of storage objects tracked, disk storage, streaming data size/row limits, etc.
  • Internal tables and result sets are limited to 2^31 rows. 
  • The buffer manager has a max addressable space of 16 terabytes - but due to fragmentation you'd expect that the max usable would be less (this is relatively easy to scale up with a larger block size when we need to).  This is the maximum amount of storage available to Teiid for all temporary lobs, internal tables, intermediate results, etc.
  • The max size of an object (batch or table page) that can be serialized by the buffer manager is 32 GB - but no one should ever get near that (the default limit is 8 MB). A batch is set or rows that are flowing through Teiid engine.
Handling a source that has tera/petabytes of data doesn't by itself impact Teiid in any way.  What matters is the processing operations that are being performed and/or how much of that data do we need to store on a temporary basis in Teiid.  With a simple forward-only query, as long as the result row count is less than 2^31, Teiid be perfectly happy to return a petabyte of data.

2. what are the soft limits for Teiid based upon the configuration such that it could impact sizing

Each batch/table page requires an in memory cache entry of approximately ~ 128 bytes - thus the total tracked max batches are limited by the heap and is also why we recommend to increase the processing batch size on larger memory or scenarios making use of large internal materializations. The actual batch/table itself is managed by buffer manager, which has layered memory buffer structure with spill over facility to disk. 

3. There are open file handle and other resource considerations (such as buffers being allocated by drivers) that are somewhat indirect from Teiid depending upon the particulars of the data source configurations that can have an impact as well.


4. Using internal materialization is based on buffer manager, it is directly dependent upon it.

5. When using XA the source access is serialized, otherwise source access happens in parallel. This can be controlled using # source threads/per user query.

Some scenarios may not be appropriate for Teiid.  Something contrived, such as 1M x 1M rows cross-join in Teiid, may not be a good fit for the vituralization layer.  But is that a real usecase where you are going to cursor over trillion rows to find what you are looking for? Is there a better targeted query? These are the kind of questions you need to be asking yourself when designing a data virtualization layer. 

Take look at query plan, command log and record the source latencies for a given query and see if your Teiid instance is performing optimally for your usecase. Is it CPU bound vs IO bound (larger source results and large source wait times). See if your submitted queries has been waiting in queue (you can check queue depth). Depending upon where you see the fallout is that is where you may need additional resources.

Our basic hardware recommendation is for smaller departmental use case is (double if you need HA or for disaster recovery) 
  • 16 Core processor
  • Minimum of 32 GB RAM
  • 100+ GB of buffer manager temp disk  (may be use of SSD based device will get better results when lot of cache miss or swapping of results)
  • Redhat Linux 6+
  • gigabit Ethernet
Then do a simple pilot with your own usecase(s) with your own data in your infrastructure with anticipated load. If you think that a DV server is totally CPU bound and queries are being delayed due to that, then you can consider adding additional cores to server or additional nodes in a cluster. Note again, to make to sure your source infrastructure is built to handle the load that DV is executing against it.

What would be really great would be sharing your hardware profiles that you selected for your Teiid environments, and techniques you used to get to the decision.

Thank you.

Ramesh & Steve.

Tuesday, October 14, 2014

Teiid 8.9 CR1 Posted

After a small delay Teiid 8.9 CR1 has been posted to the maven repository and the downloads: http://teiid.jboss.org/downloads/

A recap of the feature highlights so far with Teiid 8.9:
  • TEIID-3009 WITH project minimization - common table expressions will have their project columns minimized.
  • TEIID-3038 geoSpatial support for MongoDB translator
  • TEIID-3050 Increased Insert Performance with sources that support batching or insert with iterator.
  • TEIID-3044 Function Metadata is available through system tables and DatabaseMetaData.
  • TEIID-1910 TeiidPlatform for EclipseLink integration is now provided via the teiid-eclipselink-platform jar in maven.
  • TEIID-3119 Performance improvements in grouping and duplicate removal as well as general improvements to memory management.
  • TEIID-3156 Collation aware prevention of order by pushdown via the collationLocale translator property and the org.teiid.requireTeiidCollation system property.
In addition Ramesh has been hard at work getting in the initial commit of an OData 4 compliant layer utilizing the not yet fully released Olingo 4.  Ramesh has worked several issue for Olingo and may end up being a committer on the project - so congrats and let's hope for more cross project synergy with them as we move forward.

Expect another blog posting and more documentation around the new OData layer in the near future.

We are closing in on 100 issues addressed for this release - and should have time to address anything of importance you can find in the 10 days or so.  More than likely there will be a CR, so the final should be just in time for Halloween.

Thanks,
Steve / The Teiid Team

Thursday, September 25, 2014

8.9 Beta2 Available

8.9 Beta2 has been posted to the downloads: http://teiid.jboss.org/downloads/

This pre-release builds upon Beta1 to provide performance improvements for dup removal and any usage of our internal table structure and many new fixes.  In fact we are up to ~75 issues being addressed and will likely get close to 100 before the end of the release.

Special thanks to Joseph Chidiac for uncovering a variety of issues from Salesforce to windowing and general planning issues.  Community usage of the later pre-releases is vital to producing a quality final release.  So be sure to engage the forums and log issues as soon as you experience them.

Expect a candidate release to follow in approximately 2 weeks.  At which time we can start on the 8.10 and ideally the 9.0 Wildfly efforts.

Thanks,

Steve / The Teiid Team

Tuesday, September 23, 2014

Teiid Designer 8.6 Final Released

Teiid Designer 8.6 Final is now available our update site or download the archive.

This release focused on improving existing features and bug fixes.

Primary drivers for this release:
  • Enhance and upgrade Teiid Connection Importer
  • Added support for defining Native Query Procedure
  • REST Importer Enhancements
    • Added Dynamic Parameters
    • JSON WS support
  • Improved Security Definition for Data Roles

Check out our What's New for 8.6 article for details...

Barry LaFond
Teiid Designer Team

Monday, September 8, 2014

Teiid 8.9 Beta1

8.9 Beta1 is now available from the downloads and maven.  Feature highlights since Alpha1 include:
  • TEIID-3050 Increased Insert Performance with sources that support batching or insert with iterator.
  • TEIID-3044 Function Metadata is available through system tables and DatabaseMetaData.
  • TEIID-1910 TeiidPlatform for EclipseLink integration is now provided via the teiid-eclipselink-platform jar in maven.
With a total of 59 issues addressed so far, this continues to be primarily a fix release - but there's still time to vote for or log enhancements to make this release.

Ideally we'll still get OData 4 support via Olingo in the near term.  However it is becoming doubtful that it will make it into the 8.9 release.

Thanks again for all the community support, especially when there are infrastructure issues. We try to ensure that every forum posting gets attention, but notifications are still inconsistent at best. If something hasn't been addressed in a couple of days, feel free to re-comment on your topic.

Thanks,
Steve

Thursday, August 21, 2014

Red Hat JBoss Data Virtualization & Hortonworks Big Data Webinar Series


Discover Red Hat and Apache Hadoop for the Open Modern Data architecture

As the Enterprise's big data program matures and Apache Hadoop becomes more deeply embedded in critical operations, the ability to support and operate it efficiently and reliably becomes increasingly important. To aid enterprise in operating modern data architecture at scale, Red Hat and Hortonworks have collaborated to integrate Hortonworks Data Platform with Red Hat's proven platform technologies.
register-now-button.png
Join us in this interactive series, as we'll demonstrate how Red Hat JBoss Data Virtualization can integrate with Hadoop through Hive and provide users easy access to data.
Here's what you'll be signing up for:
  • Webinar 1: Delivering the Open Modern Data Architecture
  • Webinar 2: Red Hat JBoss and HDP: Turn your data into strategic asset (demo/deep dive)
  • Webinar 3: Red Hat and JBoss and HDP: Delivers the Data Lake (demo/deep dive)

We hope you can join us!
register-now-button.png

Wednesday, August 20, 2014

Teiid Market Place Git Repository

Introducing Teiid market place git repository, where you are free to share your project with all the Teiid community of users. (maybe sell?) Checkout https://github.com/teiid-marketplace

If you are a Teiid user, maybe you have written a translator to a source that Teiid does not offer, or maybe you gave a presentation you want to share, a slide-deck, a video or you wrote a cool utility over Teiid. Whatever it may be, now you can share with rest of the Teiid community at one market place.

Over the time, if a certain project is popular and licensing terms are acceptable by the Teiid project, then we can accept it as contribution to be maintained as a main Teiid project on a case by case basis. Or you can think of this as incubator space. When you contribute, I will make you the owner of that repository for any management purposes.

The only restriction is this needs to be a Teiid-specific project to be acceptable. I am not putting any license restrictions on what it needs to be; it can be of any license of your choosing (unless I am told otherwise). The acceptance of your project does NOT mean it is automatically promoted by Teiid developers; it is only an acknowledgement that it is a Teiid-specific project. Use these projects at your own RISK, as they have not been validated by Teiid project members unless they were written by members of the Teiid project.

I intend to provide a web page on http://teiid.org to link this repo's contents with descriptions soon. I can use some help here :)  So, if you have any Teiid utilities/projects to share, let me know.

I am still planning out what this needs to be, so if you have suggestions please let me know.

Thanks.

Ramesh..

Friday, August 8, 2014

Use Apache SOLR as Index store with Teiid

SolrTM is the popular, blazing fast open source enterprise search platform from the Apache LuceneTM project. 

In this article I show a working example, how one can start using Apache SOLR as index store for the logical views they have created in Teiid. This dramatically improves the query performance in certain situations when user query is over that logical view.


Now, combined this with any of big data sources like Hive, MongoDB, Accumulo etc. that is a success story you can take it to your organization.

Please share your comments and usecases.

Thanks

Ramesh..

Wednesday, August 6, 2014

Teiid 8.9 Alpha1 Posted

8.9 Alpha1 is now available from the downloads and maven.  Feature highlights include:
  • TEIID-3009 WITH project minimization - common table expressions will have their project columns minimized.
  • TEIID-3038 geoSpatial support for MongoDB translator
There was also an initial commit of a usage table in SYSADMIN that can be used for runtime dependency analysis. 

Saxon was updated several minor revisions, so let us know if you see any issues with SQL/XML functionality.

We're seeing a broad array of users logging issues across lots of functionality, so thanks again for all the community support. Top reporters include Cristiano Nicolai, Ivan Chan, and Mark Addleman. 

Be sure to log anything you find with the early releases as there is still plenty of time to get them addressed.

Thanks,
Steve

Monday, August 4, 2014

Teiid In JBoss Community Asylum

JBoss Community Asylum is audio podcast that feature many interesting technical talks about various community projects hosted in JBoss.org.  

Steve and Ramesh joined me last week to chat about Teiid and data virtualization. This is often a mysterious topic for many so I encourage you to go listen to the podcast episode http://in.relation.to/Bloggers/JBossCommunityAsylumWhatKindOfAnimalIsTeiid

People in Asylums can come back any time to haunt you.


Emmanuel



Tuesday, July 22, 2014

Teiid Web Console 1.2 Released

A new version of Teiid Web-Console application has been released along with Teiid 8.8 release. You can download the new version at http://sourceforge.net/projects/teiid/files/webconsole/1.2/Final/teiid-console-dist-1.2.0.Final-jboss-as7.zip/download

Using the Teiid web console you can manage/monitor and administer various aspects of the Teiid Server. The new version adds feature to enable Audit/Command/trace logging without editing the standalone-teiid.xml file manually. It also has few bug fixes.

Ramesh..

Thursday, July 17, 2014

Teiid 8.8 Final Released

We are pleased to announce the timely release of Teiid 8.8.  It is now available for download and from the Maven repository.  While we didn't quite make it to 100 issues, this release still has a good set of features and quite a few fixes.  Main features include:
  • TEIID-2647 Translator for Cloudera's Impala data warehouse has been added. See the reference for more.
  • TEIID-2902 More incremental joins where the whole dataset does not need fully sorted or even read.
  • TEIID-2263 More control over MAKE (NOT) DEP hints via syntax that allows for traversing views. See the reference for more.
  • TEIID-2263 Common Table (WITH) planning improvements such that the usage of a WITH clause in a view or inline view no longer inhibits planning as part of the larger query.
  • TEIID-2980 XMLTable Array Support - XMLTABLE may now return arrays for sequence values.
  • TEIID-1675 Updatable LDAP multi-valued attributes - using an array type mapping, LDAP multi-valued attributes can be inserted/updated.
  • TEIID-3018 System tables for view/procedure definitions - the SYSADMIN Views and StoredProcedures tables can be used to access the definitions of the views/procedures.
Thanks to everyone who helped in this release from the community - all bugs/posts are helpful in refining Teiid.  There was a lot of important OData feedback from Luca Gioppo, John Muller, Greg MacKinnon, and others.  Hopefully 8.9 will be the start of the OData v4 work - which will require all sorts of new testing.  

And now for the obligatory product plug - if you have been using Teiid and would like fully supported version, Red Hat has released Data Virtualization V6 (DV6).  The DV platform has comprehensive support from Red Hat including services/training.

Thanks,
Steve and Ramesh

Tuesday, July 8, 2014

Teiid 8.8 CR1

We are closing in on the 8.8 release with the availability of 8.8 CR1.   Notable features since Beta2 include:
  • TEIID-3018 System tables for view/procedure defintions - the SYSADMIN Views and StoredProcedures tables can be used to access the defintions of the views/procedures. 
We will also be kicking off Teiid 8.9 development shortly.  Expect more OData support in this release along with the usual performance improvements and fixes.

Please provide any feedback that you can on CR1 or vote/log issues for 8.9 and thanks again for all of the community support.

Steve

Wednesday, July 2, 2014

Kerberos authentication with Teiid

Teiid supports kerberos authentication over its JDBC and ODBC driver. It is also possible to configure OData access to use kerberos using SPNEGO with little amount of configuration.

Recently I wrote few articles while testing the feature, if interested see
  1. A practical "how-to" guide to Kerberos authentication with Teiid over JDBC
  2. A practical "how-to" guide to implement kerberos authentication with Simple REST Web App
  3. A "how to" guide to "delegate" kerberos token to REST based data source in Teiid
In article (1) shows how to configure Teiid to participate in Kerberos authentication.

In article (2) I showed a example to create simple web-app to configure with SPNEGO to participate in Kerberoes.

In article (3) I showed an example, to delegate the logged in user into Teiid, to the REST web-service as passthrough with out extra negotiation. 


Thanks

Ramesh..

Monday, June 23, 2014

Teiid 8.8 Beta2

Teiid 8.8 Beta2 is now available from the downloads.  Features since Beta1 include:
  • TEIID-2980 XMLTable Array Support - XMLTABLE may now return arrays for sequence values.
  • TEIID-1675 Updatable LDAP multi-valued attributes - using an array type mapping, LDAP multi-valued attributes can be inserted/updated.
Over 70 issues have now been addressed in the release so we are expecting a high quality result by mid-July. Please provide feedback as early as possible so that the changes make it into the CR build.

Thanks again for all the feedback and forum questions. A general infrastructure issue that we are seeing is that there seem to be periods of time when we are not getting forum notifications. We do ordinarily strive to respond as quickly as possible.

Thanks,
Steve

Monday, June 16, 2014

Teiid Designer 8.5 Final Released

Teiid Designer 8.5 Final is now available our update site or download the archive.

The primary drivers for this release were:
  • Replace multiple Teiid client plugins with single plugin
    • greatly improves ability to support future Teiid runtimes
  • Upgrade Teiid runtime compatibility to version 8.7
  • Expose Translator import properties within Teiid Connection importer
  • Support for dynamic extension metadata (custom translators)
  • VDB Editor layout improvements
  • Make System functions available for use at design-time

Check out our What's New for 8.5 article for details...

Barry LaFond
Teiid Designer Team

Monday, June 9, 2014

Teiid 8.8 Beta1

Teiid 8.8 Beta1 has just been posted to the downloads.  Notable features since alpha1 include:
  • TEIID-2263 More control over MAKE (NOT) DEP hints via syntax that allows for traversing views.  See the reference for more.
  • TEIID-2263 Common Table (WITH) planning improvements such that the usage of a WITH clause in a view or inline view no longer inhibits planning as part of the larger query.
There has also been quite a bit of general issues addressed (including several with MongoDB), so this should be a fairly stable pre-release.  We have also seen that the Resteasy patch is now also required for SOAP WS translator/resource adapter operations, so please apply the patch proactively.

Thanks again for all the feedback.  Be sure to let us know of any issues as there is still plenty of time to get them addressed before the final release.

Thanks,
Steve

Monday, May 12, 2014

Docker and Red Hat JBoss Data Virtualization (Teiid)

Seems like every one is looking into Docker now a days and JBoss Data Virtualization is no exception. Take look at the following article and see how you can stand up JDV Docker image for your distribution or testing

http://www.ossmentor.com/2014/05/docker-and-red-hat-jboss-data.html

Note: JBoss Data Virtualization (JDV) is product version Teiid that Redhat sells subscription to.

Now, to a bigger question I would like to engage community is

1) is there any interest in bring this to Teiid community?
2) What configurations?
3) How do you intend to use it?
4) Would you like us to build image every time we do a Teiid release and update the Docker.io repository along with Teiid download section?

Some other ideas with docker

1) What would be really great is somebody help us with building quick start examples with Docker images, then we can reduce the barrier of using data virtulaization technology.

2) Build Docker images so that one deploy multiple instances of it then try to use them in performance testing to measure metrics

Any volunteers? this got to be pretty exciting assignment to learn and contribute back to community. Most of it is already in above article -:) Gosh I want to do it, wait.. I have other features I am working on for Teiid can't get to it now.

Thanks,

Ramesh..

Sending Command and Audit Logging to Database

If you are interested in sending the Teiid's Command or Audit logs to a database instead of file, we have written logging handler for those. Please follow the instructions in this article https://community.jboss.org/wiki/CommandAuditLoggingToDatabase to configure your system.

Ramesh..

Thursday, May 8, 2014

Teiid 8.8 Alpha1 Posted

8.8 Alpha1 is now available from the downloads and maven.  Feature highlights include:
  • TEIID-2647 Translator for Cloudera's Impala data warehouse has been added. See the reference for more.
  • TEIID-2902 More incremental joins where the whole dataset does not need fully sorted or even read.
Ramesh has added command and audit logging to a variety of database targets through JPA - see TEIID-2938 and Logging for more.

Generally this will be a fix and enhancement focused release, but there still is time to propose additional feature work.

Thanks again for all the community support,
Steve

Wednesday, April 16, 2014

Teiid 8.7 Final Released

Teiid 8.7 has been released and is now available for download and from the Maven repository.  This was both a rich feature and a fix release with over 100 issues addressed.

Main features in this release include:
  • TEIID-1070 Amazon SimpleDB support has been contributed by Radim Hopp. See the Admin and Reference Guides for more. See example here
  • TEIID-2761 Security Enhancements added support for a grant-all attribute on a role. Also added the ability to specify the security domain for a VDB. See the reference guide for more.
  • TEIID-2780 Apache Solr Translator has been added. See the Admin and Reference Guides for more. See example here
  • TEIID-2809 Pattern based system property substitution has been added to VDB.XML files. Only supported if the DVB is being deployed JBoss EAP.
  • TEIID-2844 Remove unauthorized columns from SELECT * can be enabled via the session property ignore_unauthorized_asterisk. See the Reference for more.
  • TEIID-2863/TEIID-2761 More robust GSS-API support. This issue also implements a feature where a user can now define a "security-domain" per VDB. User no longer has to depend upon the transport level authentication.
  • TEIID-2826 Revamped build structure for translators and connectors so that it is easier to package them and deploy them to a Teiid server.
  • TEIID-2859 TEXTTABLE FOR ORDINALITY a FOR ORDINALITY column can now be used with TEXTTABLE. See the Reference for more.
  • TEIID-965 Apache POI based Excel connectivity has been added. See the reference for more. See example here
  • TEIID-2888 Designer VDB support in Embedded is now available. With a few caveats, Designer .INDEX based vdbs can now be used with Teiid Embedded. See the Embedded Guide for more.
  • TEIID-2904 Translator Properties Translator import, override and extension-metadata properties are can retrieved through Admin API
  • TEIID-2627 Accumulo Translator has been added. See Admin and Reference guides for more information. See example here
We sincerely thank everyone who participated in the community.  All bugs/posts are helpful in refining Teiid.  We are looking forward to your input on 8.8.

If have been using Teiid and would like fully supported version, Red Hat has released Data Virtualization V6 (DV6).  The DV platform has comprehensive support from Red Hat including services/training.

Thanks,
Steve and Ramesh

Friday, March 28, 2014

8.7 CR1 Posted

8.7 CR1 has been posted to the downloads.  Since Beta2 this release includes features such as:
  • TEIID-2859 TEXTTABLE FOR ORDINALITY a FOR ORDINALITY column can now be used with TEXTTABLE. See the Reference for more.
  • TEIID-965 Apache POI based Excel connectivity has been added. See the Reference for more.
  • TEIID-2888 Designer VDB support in Embedded is now available. With a few caveats, Designer .INDEX based vdbs can now be used with Teiid Embedded. See the Embedded Guide for more. 
There is also a new framework for translator to supply importer and extension metadata for better integration with Designer.  This will likely evolve slightly before the final release.  Also there is a small KI with this release affecting custom delegating translators.  If you experience an issue with a custom delegating translator, please follow master or wait for CR2.

In total we are closing in on 100 issues for this release and anticipate a release date around 4/16.  Thanks for all of the great feedback, please do check out the candidate releases as we still have time to refine/add fixes before final.

Teiid 8.8 will be kicking off shortly with the CR out of the way.  Take a pass through JIRA or log additional issues if there are things that you want to see make it into the next release.

Thanks,
Steve

Thursday, March 20, 2014

Read Microsoft Excel Document As Relational Table Using Teiid

If you are thinking, who in this age of big data with Hadoop, MongoDB et.al one still bothers to fiddle around with Excel? In reality there are still lot of corporate users out there, do their analytics and reporting using Excel. So, this is still is very important source of data.

From very early versions of Teiid, Teiid supported consuming the data from Microsoft Excel documents, however it always relied upon the "Excel" ODBC driver that came built in Windows platforms. Teiid then used "jdbc-odbc" bridge driver provided by Oracle's JDK to read the data in. Although this solution worked most of the time, it was very tedious and sprinkled with issues and limited the access to Windows platform.

In latest version of Teiid 8.7, a new translator is introduced which is based Apache POI framework, which gives platform independent way to read Excel documents. This is worked out to be much more simple and also gave a way define the metadata for the data in Excel Documents.

If you are interested, I wrote a step-by-step example for this here https://community.jboss.org/wiki/MicrosoftExcelDocumentIntoRelationalTable

Ramesh..

Monday, March 17, 2014

Making business intelligence systems more agile with data virtualization By Rick van der Lans

This is excellent opportunity for all customers, prospects and partners to hear from industry expert and analyst Rick van der Lans as he discusses the IT strategy to build agile business intelligence systems and transforming data into competitive advantage using Jboss data virtualization middleware.  This session will provide great insights into leveraging data virtualization technology to improve delivery of business intelligence systems and data warehouse utilization.  

Time: Wednesday, March 19, 2014 | 15:00 UTC | 11 a.m. (New York) / 4 p.m. (Paris) / 8:30 p.m. (Mumbai)

About Rick van der Lans, managing director, R20/ConsultingRick is an independent consultant, author, and lecturer specializing in data warehousing, business intelligence, big data, data virtualization, and database technology. He has written the industry's first independent book on data virtualization titled, “Data Virtualization for Business Intelligence Systems: Revolutionizing Data Integration for Data Warehouses.” Rick has advised many large companies worldwide on defining their data warehouse, business intelligence, and SOA architectures. He is the chairman of the European Enterprise Data and BI Conference and writes regularly for B-eye-Network.com.

http://app.engage.redhat.com/e/es.aspx?s=1795&e=332131&elq=24680766e7f746ac85a30a30deed2454

Wednesday, March 12, 2014

Teiid 8.7 Beta2

Teiid 8.7 Beta2 is released today. The features since Beta 1 include

  • TEIID-2863/TEIID-2761 This issue fixes the previous issues with GSS-API in ODBC transport. This issue also implements a feature where a user can now define a "security-domain" per VDB. User no longer has to depend upon the transport level authentication.
  • TEIID-2870: Join types that are allowed and pushed to source can be set on the Foreign Key as metadata or allow them to use federated join.
  • TEIID-2826: revamped build structure for translators and connectors, so that it is easier to package them and deploy them to Teiid server
Many other bugs have been fixed. We may be in last couple weeks of development on 8.7 version, so start testing your usecases and let us know any small features we can sneak it in. Once we get to CR1 (which may be next) then there will be no more feature work, we will only work on bug fixes from then on to 8.7 Final (in 3~4 weeks).

Thank you.

Ramesh..

Teiid Designer 8.4.0 Final Released

Teiid Designer 8.4 Final is now available our update site or download the archive.

The primary driver for this release was to upgrade the Teiid client compatibility to include Teiid 8.6 release as well as working a number of bug fixes and a few new features and improvements.

Check out our What's New for 8.4 article for details...

Barry LaFond
Teiid Designer Team

Thursday, March 6, 2014

Teiid on OpenShift

Are you looking for solution where Teiid deployed on PaaS solution like OpenShift? Please look at below articles that show two separate examples n OpenShift.

1) Provision Data Virtualization on OpenShift and Connect from Teiid Designer
https://community.jboss.org/wiki/ProvisionDataVirtualizationOnOpenShiftAndConnectFromTeiidDesigner

2) Salesforce as a REST Service using Data Virtualization
https://community.jboss.org/wiki/SalesforceAsARESTServiceUsingDataVirtualization

Please take a test drive, let us know your comments.

Thanks

Ramesh..


Tuesday, February 25, 2014

Teiid Designer 8.4 Beta1 available

Work on Teiid Designer 8.4 release is nearing completion. Get the latest milestone, 8.4 Beta1 via via our update site or download the archive.

The primary driver for this release is to upgrade the Teiid client compatibility to include Teiid 8.6 release. Check out our What's New for 8.4 article for details on some of the new features and improvements.

Barry LaFond
Teiid Designer Team

Tuesday, February 18, 2014

Teiid 8.7 Beta1

After a minor weather delay, we're back on track with 8.7 pre-releases with Beta 1.  Feature highlights since Alpha1 include:
  • TEIID-2809 Pattern based system property substitution has been added to VDB.XML files. Only supported if the DVB is being deployed JBoss EAP.
  • TEIID-2844 Remove unauthorized columns from SELECT * can be enabled via the session property ignore_unauthorized_asterisk. See the Reference for more.
The pace of community issues has been good, so we're up to 55 issues addressed to date. 

At this point we should miss the mid-March release date slightly, but we may need to lengthen the cycle for feature work.  That means the release will be out any time between the end of March to the beginning of May.  Should it be needed, we'll work toward an 8.6 cumulative patch for those who wish to stay on a release version.
  
Thanks,
Steve

Friday, February 14, 2014

Teiid Designer 8.4 Alpha1 available

Work on Teiid Designer 8.4 release is progressing nicely and the first development release, 8.4 Alpha1 is available via update site.

The primary driver for this release is to upgrade the Teiid client compatibility to include Teiid 8.6 release.  This includes UI support for:
  • Added a native type property to procedure OUT parameters - TEIIDDES-1863
  • Ability to define a virtual table as global temporary table - TEIIDDES-1794
In addition, we're include a number of bug fixes and minor enhancements to importers as well as improvements to VDB validation.

Barry LaFond
Teiid Designer Team


Tuesday, February 11, 2014

JBoss Data Virtualization v6 GA Released

Have you been waiting for productized version Teiid 8.x? Then your wait is over, see below for announcement from Redhat.

Redhat is excited to announce the general availability and launch of JBoss Data Virtualization v6.

Effective data utilization is top priority on every IT and business executive agenda:
Data volume, velocity and variety is growing exponentially. Most of your customers are drowning in data and struggling to gain actionable information. Its time to show them how to build efficient data supply chain instead of data warehouse to gain competitive advantage.
Red Hat JBoss Data Virtualization, enables organizations to deliver timely, actionable, and unified information through lean integration of data spread over multiple applications and technology silos including Hadoop and traditional data warehouses. Red Hat JBoss Data Virtualization enables agile data utilization in 3 easy steps:

  • Connect: Access data from multiple, heterogeneous data sources.
  • Compose: Easily combine and transform data into reusable, business-friendly virtual data models and views.
  • Consume: Make unified data easily consumable through open standards interfaces.

PR release: http://www.redhat.com/about/news/press-archive/2014/2/red-hat-launches-jboss-data-virtualization-6

Launch blog: http://www.redhat.com/about/news/archive/2014/2/build-data-supply-chains-not-data-warehouses


Ramesh..

Monday, January 13, 2014

Integrate Apache Solr search results with enterprise data with Teiid

Apache Solr is a full text search engine built on top of Apache Lucene for indexing and searching any data contents. This is been widely used by many organisations for their searching needs.

Typically when a user wants to use Apache Solr, they will configure an instance of Apache Solr or SolrCloud with relevant indexing fields and publish all their data/documents that instance to be indexed. Once the documents are indexed, they can use Solr APIs to search the documents and use it for their purposes. This is where I believe Teiid can help ease the uploading of the enterprise data and with integration of search results with other data sources.

Starting with Teiid 8.7.Alpha1, Teiid provides translator for Apache Solr. Using this translator, user can 

  1. Continually add/update the documents in the search system from other sources automatically.
  2. If the search fields are stored in Solr system, this can be used as very low latency data retrieval for serving high traffic applications
  3. Solr translator be used as fast full text search, where the Solr document only contained the index information, then use the results as inverted index to gather target full documents from the other enterprise sources such as RDBMS, Web Service, SalesForce etc, all in single client call transparently with out any coding.
To illustrate this integration in an example I wrote this article, please read on.. Integrate Apache Solr with Teiid

Let us know if you have any comments or questions in Teiid forums.

Thanks.

Ramesh..

Sunday, January 12, 2014

Teiid 8.7 Alpha1

After the holiday slowdown we're getting right back into the swing of things with the first 8.7 prerelease.  Feature highlights include initial commits of the following connectivity:
  • TEIID-1070 Amazon SimpleDB support has been contributed by Radim Hopp.  See the Admin and Reference Guides for more.
  • TEIID-2780 Apache SOLR support has been contributed by a collaboration of Mike Walker, Jason Marly, and Ramesh.
  • TEIID-2627 Accumulo support has been added via a new translator/resource adapter.
This is a fantastic amount of new connectivity for Teiid - and we're not even to the first beta.  Together with some other features and lots of fixes we're looking forward a strong 8.7 release.

Any additional testing of the new sources or just early feedback in general is greatly appreciated.

Thanks,
Steve