Many Essbase customers regularly extract Essbase outline metadata for a variety of reasons. One common reason companies use outline metadata is for building dimensions in other Essbase databases. In earlier versions of Essbase, there was no good way to extract the outline information to a usable format. Starting in Essbase 11.1.2.x, the Maxl scripting language introduced the capability to export outline information to an XML format, but Oracle has not yet created any tools that consume the XML in its native format. As such, there is a need for other outline extraction tools such as the recently released Next Generation Outline Extractor.
The Next Generation Outline Extractor is a complete rearchitecture and rewrite of the original OlapUnderground Essbase Outline Extractor. The original Extractor was written in Visual Basic, which limited its use to Windows. The Next Generation Outline Extractor is written using the Essbase Java API, which allows it to run on a number of operating systems and opens up the possibility of new functionality. One interesting new feature of the Next Generation Outline Extractor is the ability to export outline metadata to a relational database. In this article, we will explore the process for extracting an Essbase outline to a relational database and configuring the extraction process to work from a command file that will allow for automation.
Note: The Next Generation Outline Extractor is a free utility written, maintained, and supported by Applied OLAP as a service to the Essbase community and has experienced tens of thousands of downloads.
Creating the Initial Extract
The Next Generation Outline Extractor features both a GUI interface and a command-line interface so users can choose the method they prefer. To create the initial extract, it is almost always easier to use the GUI and its wizard interface to create a properties file. In this case, I am launching the GUI interface from the Next Generation Outline Extractor root directory by clicking the “gui-olapunderground-outline-extractor.cmd” file.
Next, on the Start tab of the dialog, select the option to create a new extract, and then press the Next button.
The next tab in the dialog is where you provide your credentials for login and choose the Essbase cube you want to export. Once you have completed this step, hit the Next button.
The third tab of the dialog allows you to specify dimensions and member filters for members you explicitly want to include, or exclude, from the extraction. In my case, I want to export everything, so I simply press the Next button.
At the Output Handlers step, choose the Relational Cache Writer. Note that once you have selected this writer option, the Relational Options tab becomes enabled. Press the Next button to continue.
Configure the relational connection using standard JDBC connection information. Once you have entered and tested the connection information, press the Next button to continue.
Note: Due to licensing restrictions, the Next Generation Outline Extractor ships with the JDBC drivers for Microsoft SQL Server, but not with the JDBC drivers for Oracle. To export to Oracle, or to any other JDBC-compatible database, download JDBC drivers for the relevant database, place the files into the Next Generation Outline Extractor lib subdirectory, and then use the connection information from the documentation in the dialog above.
The Relational Cache Writer does not have options, so you can skip the Options dialog. Press the Next button to continue.
The performance tab gives you an opportunity to tune the extraction algorithms. This tab is generally used to adjust the member query delay, which slows down calls to the Essbase server in order to minimize the number of ports used. The Essbase API uses an extraordinary number of ports and can easily exhaust the available ports on the server. (For more information on port exhaustion, see my blog entry at http://timtows-hyperion-blog.blogspot.com/2007/12/essbase-api-error-fix-geeky.html.)
Once the performance options have been set, press the Next button to continue.
The final step is to save the properties file so you can use it with the command line extractor. To save a properties file, press the Save button on the Finish tab, name the file, and then press the Save button.
Relational Schema
The relational schema created by the Next Generation Outline Extractor creates a total of five tables. Four of the tables contain member information in a highly normalized format. As the extractor database supports multiple versions of an outline in the same relational database, the fifth table contains extraction version information. Here is the schema:
The relational extraction requires that the credentials used for the relational login either have authority to create and drop tables or the database tables must be previously created in the database.
Automating the Extraction
At this point, you have a properties file containing the settings you need to run the extract in command-line mode. In fact, to set up the command file, simply modify the olapunderground-outline-extractor.cmd file to point to the properties file. The path must be specified using forward slashes as the path separator as in the example below:
%_JEXEC% -jar olapunderground-outline-extractor.jar -properties "D:/Temp/Next Generation Outline Extractor/olapunderground-outline-extractor.11.1.2.3/Sample.Basic.relational.extract.properties"
Note: Don’t forget that you may need to modify your command files to modify the JAVA_HOME setting, located at the top of the file, as shown below. Note the location where you have the Java runtime installed may differ.
set JAVA_HOME=C:\Progra~2\Java\jre7
Extraction Results
Once the extraction is complete, the database tables will contain information from your Essbase outline. The CACHED_OUTLINE_VERSIONS table contains one record for each extraction attempt. The key fields in this table include the ID, OUTLINE_INFO_ID and ACTIVE_FLAG fields. The ID is a unique, sequential number that identifies an extract. The OUTLINE_INFO_ID field is the text field that combines the selected server, Essbase application name, and Essbase cube name to identify the outline. The ACTIVE_FLAG field indicates the success of the extraction attempt; an attempt that has an ACTIVE_FLAG value of 1 completed successfully.
At the time of this writing, the CACHED_OUTLINE_VERSIONS table also includes several fields that are not used. These fields, CHECKSUM, CONNECTION_KEY_PART, and OUTLINE_INFO_ID are used in the Dodeca Spreadsheet Management System from which the relational extract was adapted. They may be removed in future versions of the Next Generation Outline Extractor.
The CACHED_OUTLINE_MEMBERS table contains one record for each member extracted from the outline and includes fields for most properties that are available for a member. The ID column in this table is a unique, sequential number that identifies a member. The VERSION_ID column is a foreign key used to identify the extract attempt identified by the ID column in the CACHED_OUTLINE_VERSIONS table.
The remaining tables, CACHED_OUTLINE_MEMBER_ALIASES, CACHED_OUTLINE_MEMBER_UDAS, and CACHED_OUTLINE_MEMBER_ATTRIBS, may also contain records if your outline has aliases, UDAs or member attributes. The MEMBER_ID field in each of these tables is a foreign key to the ID field in the CACHED_OUTLINE_MEMBERS table which has a one-to-many relationship with the tables containing the aliases, UDAs, and attributes.
To query the extracted data, join the version table, members table, and alias, UDA, and/or attribute table as in the following example:
SELECT m.MEMBER_NAME as 'Name',
a1.ATTRIBUTE_NAME as 'Population’
FROM CACHED_OUTLINE_MEMBERS m,
CACHED_OUTLINE_MEMBER_ATTRIBS a1
WHERE m.ID = a1.MEMBER_ID
AND m.DIMENSION_NAME = 'Market'
AND a1.ATTRIBUTE_DIMENSION = 'Population'
AND M.SHARE_FLAG = 0
AND m.VERSION_ID = (SELECT MAX(ID) FROM CACHED_OUTLINE_VERSIONS
WHERE OUTLINE_INFO_ID = 'tigercat|Sample.Basic'
AND ACTIVE_FLAG = 1)
ORDER BY m.MEMBER_NUMBER
Summary
The Next Generation Outline Extractor has a number of very useful features, including the ability to export outline metadata directly to a relational database and the ability to run the extraction process from the command line. I hope you are able to use the Next Generation Outline Extractor to make your job easier.
