How can I access to the stored metadata of "Documents and Media" documents?

Issue

I am trying to develop a module that manages files and metadata in the Documents and Media Library but I am struggling to interact with the metadata from metadata sets.

How can I access the stored metadata of Documents and Media documents?

Can you please provide us with the correct classes and services to access them?

Environment

• DXP 7.4

Resolution

Here there is some information about the stored metadata of "Documents and Media" documents and how to access it.

1. Database tables:

The database tables related to "Documents and Media" documents and their metadata are:

• DLFileEntry, DLFileVersion, DLFileEntryType, DDMStructure, DLFileEntryMetadata, DDMStorageLink, DDMField and DDMFieldAttribute

The relation between these tables are:

1. Every document is stored in the DLFileEntry. table. The documents can have several document versions in the table DLFileVersion.
2. The metadata definition of a "Document Type" is stored in DLFileEntryType (name) and DDMStructure (structure). DLFileEntryType links DLFileVersion with DDMStructure
3. If the DLFileVersion version has any "Document Type" with metadata there will be an entry in DLFileEntryMetadata, that links DLFileEntry (dlFileEntryId), DLFileVersion (dlFileVersionId), the DDMStructure and the DDM data storage (ddmStorageId)
4. In DXP 7.4, the data storage of DLFileEntryMetadata is stored in DDMStorageLink, DDMField, and DDMFieldAttribute

Important: You must ALWAYS modify these tables using the Liferay API. DO NOT update/delete the table records using any other mechanism.

2. Java code to read metadata from a document version:

As a starting point, I have implemented the groovy script getMetadata-AllFields-DLFileEntry_74x.groovy which retrieves all the metadata from all the DLFileEntry objects of the database and it dumps them in the script output.

To execute the script you have to go to Control Panel => Server Administration => Script and copy the content of the attached file.

The Java classes related to these tables are located in the following java packages:

• Model: com.liferay.document.library.kernel.model.* => DLFileEntry, DLFileVersion, DLFileEntryMetadata, DLFileEntryType:
• Services: com.liferay.document.library.kernel.service.* => DLFileEntryLocalService, DLFileVersionLocalService, DLFileEntryMetadataLocalService, DLFileEntryTypeLocalService:

You will also have to use the Dynamic Data Mapping classes to get all the information from DDMField, DDMFieldAttribute, DDMStorageLink, and DDMStructure, for more information, see: 

• https://github.com/liferay/liferay-portal/tree/master/portal-kernel/src/com/liferay/dynamic/data/mapping/kernel

When you want to get the metadata of a document, you usually follow this path:

1. Get the DLFileVersion. You can get it from the DLFileEntry calling dlFileEntry.getFileVersion() or using the DLFileVersionLocalService service
2. Get the DLFileEntryType (= document type) from the DLFileVersion, just calling dlFileVersion.getFileEntryTypeId()
3. Get the DDMStructures (= document type definition)  from DLFileEntryType, calling dlFileEntryType.getDDMStructures()
4. Once you have the DDMStructures and the DLFileVersion, you can get the DLFileEntryMetadata by calling the API: DLFileEntryMetadataLocalServiceUtil.getFileEntryMetadata(ddmStructure.getStructureId(), dlFileVersion.getFileVersionId())
5. With the DLFileEntryMetadata object you can access the actual data from the DDM data storage by calling StorageEngineManagerUtil.getDDMFormValues(fileEntryMetadata.getDDMStorageId())
6. This call returns a DDMFormValues object where you can get the stored DDMFormFieldValue objects.

3. Java code to add/update metadata to a document version:

Here there are some Liferay code examples where the product is inserting metadata, use it as an example to write your own code:

3.1 DLFileEntry creation: 

See the following code: https://github.com/liferay/liferay-portal/blob/7981e83f8002dec2719622319ec5efc72b279042/portal-impl/src/com/liferay/portlet/documentlibrary/service/impl/DLFileEntryLocalServiceImpl.java#L2092-L2096 here _addFileVersion method calls _dlFileEntryMetadataLocalService.updateFileEntryMetadata to store the data stored in the ddmFormValuesMap map

3.2 DDMFormValues creation:

You have some examples of how the ddmFormValuesMap map is created here:

• https://github.com/liferay/liferay-portal/blob/7981e83f8002dec2719622319ec5efc72b279042/portal-impl/src/com/liferay/portal/repository/liferayrepository/LiferayRepositoryBase.java#L107-L139

You have two examples of how DDMFormValues is created here:

• https://github.com/liferay/liferay-portal/blob/7981e83f8002dec2719622319ec5efc72b279042/modules/apps/document-library/document-library-web/src/main/java/com/liferay/document/library/web/internal/portlet/action/EditFileEntryMVCActionCommand.java#L1188-L1215
• https://github.com/liferay/liferay-portal/blob/7981e83f8002dec2719622319ec5efc72b279042/modules/apps/portal/portal-tika/src/main/java/com/liferay/portal/tika/internal/metadata/TikaRawMetadataProcessor.java#L138-L170

3.3 Update file metadata:

There is another example where automatically extracted metadata from the document is added to the document, see RawMetadataProcessorImpl and TikaRawMetadataProcessor classes:

• https://github.com/liferay/liferay-portal/blob/release-2023.q3/portal-impl/src/com/liferay/portlet/documentlibrary/util/RawMetadataProcessorImpl.java

However, this is a special case, as it uses an internal structure called RawMetadataProcessor that is not available in the user interface.