Understanding Data Extraction

Business models describe the data created, captured, and leveraged to perform business functions. Liferay empowers users to design custom models with objects and seamlessly integrate them with Liferay's core frameworks. Once a model's definitions, relationships, and individual entries are designed, exporting and importing this data between environments is critical for boosting operational effectiveness.

Migrating data through exporting and importing

• Reduces manual efforts.
• Eliminates mistakes.
• Promotes data consistency.
• Frees up valuable time and resources.

Liferay provides multiple methods leveraging batch APIs to perform efficient and reliable data transfer between environments, including headless APIs, the data migration center, and batch client extensions.

Understanding Batch APIs

Liferay's batch APIs empower developers to manage data programmatically. These APIs include batch processing methods which enable you to export or import multiple items within the same API call through the Batch Engine. For large-scale migrations or integration with external systems, this batch processing capability provides an efficient, automated method for extracting multiple definitions in single API requests.

Batch is considered the official mechanism to migrate object data, and should be considered the primary mechanism for exporting and importing data into environments. Throughout this course, you'll learn about migrating data with batch client extensions.

Selecting Strategies for Extracting Data

JSON records are not typically created by hand – instead, models can be created with Liferay's available UI tools and APIs. Once crafted, those definitions can be exported and packaged into deployable assets for other environments. Efficient transfer of information that will reside in a JSON's data block depends on exporting. While a deep dive into data migration strategies is beyond this course's scope, understanding available data preparation approaches and batch strategies is crucial to appropriately set up data for batch client extensions.

Liferay tooling simplifies the process of preparing data to migrate with batch client extensions. To extract data and prepare it for migration, you can leverage

• UI Exports
• The Data Migration Center
• Headless APIs

Each of these options has its own strengths when leveraged with client extensions, however each method has its own benefits and techniques specific to their usage.

UI Exports

The simplest method for exporting completed object models is the Liferay UI. Within their respective applications, you can export assets such as objects as a JSON file:

When you export object definitions, the JSON includes the object's complete schema, including configurations, fields, actions, relationships, and more. You can also export entire object folders to include multiple, related object definitions in the same JSON file. Exporting through Liferay's UI leverages the same mechanisms as batch APIs to package the data into the downloaded JSON.

When exporting through the UI, it's important to keep these considerations in mind:

• Currently, you cannot export notification templates via the Liferay UI.
• The JSON for objects only includes one-to-many relationships if the object is on the parent side.
• Exported models will not include the configuration block or the items array by default. You'll learn more about these blocks later in this module.

Data Migration Center

The Data Migration Center performs import and export operations with Liferay Objects schemas and entries. To generate the necessary structure for batch client extensions, you should choose the jsont (Technical JSON) export format:

Exporting using this method will generate batch configurations in addition to the item information in the data block. However, the resulting data may need the following modifications to be imported successfully:

• containsHeaders should be set to true
• createStrategy should be set to UPSERT
• The code and label_i18n properties should be added to the status object
• If directly exporting an object, add missing properties to any auxiliary fields

Headless APIs

Directly leveraging Liferay's headless APIs is another available data extraction approach. You can call these APIs through curl, third party API platforms, and Liferay's API Explorer. Browse available APIs through Liferay's API Explorer, located in the REST Services menu at [server]:[port]/o/api (e.g., localhost:8080/o/api).

For bulk data manipulation, Liferay recommends leveraging batch APIs directly or through a batch client extension to benefit from the more efficient, high-performance batch engine. The Foundations of Liferay's Headless API course discusses headless APIs in greater detail (coming soon!).

Best Practices for Exporting

When approaching data extraction with batch client extensions, it's crucial to carefully consider which export strategy to use (and when). To help make this decision, consider these best practices:

• Plan Strategies in Advance: Though you can use each of these export tools together, carefully consider which options are best for the processes you're implementing. This often depends on the complexity of the data, the level of interaction required, and the customization needed.
• Assign Meaningful ERCs: By default, Liferay assigns a random UUID to object folders and definitions. Assign consistent, human-readable ERCs before exporting object definitions to enhance user experience during data migration.
• Include All Data in Definitions and Picklists: Importing and overwriting existing object definitions or picklists with extracted data is permanent. To prevent loss, retain complete definitions within extracted JSON data, encompassing attributes, relationships, and any other relevant information.
• Extract with Appropriate Techniques: Leverage APIs to fetch data when appropriate, using batch APIs for bulk data extraction. Alternately, other approaches may be more efficient when extracting information from databases (where utilizing JDBC or ODBC connectors is more appropriate) or reading data from files (where data can be extracted directly from the CSV, JSON, or other formats).

Planning Clarity's Data Extraction

Clarity will leverage batch client extensions to migrate objects from lower environments to production environments. For the distributor management app, Clarity wants to ensure object definitions are refined before importing them elsewhere. Clarity adheres to this best practice by creating data models within the UI of testing environments, exporting those definitions, then packaging the JSON files into deployable assets for other environments. To extract these models, Clarity will export object definitions through the UI.

Conclusion

Liferay's efficient data migration capabilities promotes data consistency and frees up valuable time and resources. Learning techniques for extracting data with Liferay is crucial for selecting the tools suitable to your specific migration needs. Exporting data from Liferay is foundational towards reliably transferring models and other data between environments through backend client extensions.