eu.trentorise.opendata.jackan.dcat

Class DcatFactory

java.lang.Object

eu.trentorise.opendata.jackan.dcat.DcatFactory

Direct Known Subclasses:

GreedyDcatFactory

public class DcatFactory
extends Object

Factory to generate Dcat objects from Ckan ones. Conversion is done according to this mapping in Ckanext-dcat repository. In most cases this mapping is deliberately a loose one, for instance, it does not try to link the DCAT publisher property with a CKAN dataset author, maintainer or organization, as the link between them is not straight-forward and may depend on a particular instance needs. To extract more stuff during conversion, you can use GreedyDcatFactory or extend this class and override the extract* and/or postProcess* methods.

Since:

0.4.1

Author:

David Leoni

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	DESCRIPTION
protected static String	ISSUED
protected static String	MODIFIED
protected static String	TITLE
protected static String	URI_FIELD

Constructor Summary

Constructors

Constructor and Description
DcatFactory() Creates a factory with default configuration.

Method Summary

Methods

Modifier and Type	Method and Description
protected String	extractAccessUrl(CkanResource resource)
protected String	extractAccrualPeriodicity(CkanDataset dataset)
protected int	extractByteSize(CkanResource resource)
protected eu.trentorise.opendata.traceprov.dcat.VCard	extractContactPoint(CkanDataset dataset)
protected eu.trentorise.opendata.commons.Dict	extractDescription(CkanDataset dataset, Locale locale)
protected eu.trentorise.opendata.commons.Dict	extractDescription(CkanResource dataset, Locale locale)
protected String	extractDownloadUrl(CkanResource resource)
protected <T> T	extractField(CkanDataset dataset, String field, Class<T> toClass)
protected <T> T	extractField(CkanDataset dataset, String field, com.fasterxml.jackson.core.type.TypeReference<T> toType) Tries to extract a string field from a CkanDataset and casts it to target type
protected String	extractFieldAsNonEmptyString(CkanDataset dataset, String field) Like extractFieldAsString(CkanDataset, String) but also checks for trimmed non-emptiness.
protected String	extractFieldAsNonEmptyString(CkanResource resource, String field) Like extractFieldAsString(CkanResource, String) but also checks for trimmed non-emptiness.
protected Object	extractFieldAsObject(CkanDataset dataset, String field) Searches a field in CkanDatasetBase.getOthers() and then in CkanDatasetBase.getExtras().
protected String	extractFieldAsString(CkanDataset dataset, String field) Searches a field in CkanDatasetBase.getOthers() and then in CkanDatasetBase.getExtras().
protected String	extractFieldAsString(CkanResource resource, String field) Searches a field in CkanResourceBase.getOthers().
protected String	extractFormat(CkanResource resource)
protected String	extractIdentifier(CkanDataset dataset)
protected String	extractIssued(CkanDataset dataset)
protected String	extractIssued(CkanResource resource)
protected List<String>	extractKeywords(CkanDataset dataset)
protected String	extractLandingPage(CkanDataset dataset)
protected List<Locale>	extractLanguages(CkanDataset dataset) i.e.
protected String	extractLicense(CkanResource resource, String license)
protected String	extractMediaType(CkanResource resource)
protected String	extractModified(CkanDataset dataset)
protected String	extractModified(CkanResource resource)
protected eu.trentorise.opendata.traceprov.dcat.FoafAgent	extractPublisher(CkanDataset dataset, Locale locale) Notice this extractor will mostly look for special dcat fields in dataset, without resorting to ckan group, organization or maintainer as fallback.
protected String	extractRights(CkanResource resource)
protected eu.trentorise.opendata.traceprov.geojson.GeoJson	extractSpatial(CkanDataset dataset)
protected eu.trentorise.opendata.commons.PeriodOfTime	extractTemporal(CkanDataset dataset) Returns a string with values trying to respect ISO 8601 format for time intervals: https://en.wikipedia.org/wiki/ISO_8601#Time_intervals
protected List<eu.trentorise.opendata.traceprov.dcat.SkosConcept>	extractThemes(CkanDataset dataset, Locale locale, String catalogUrl) Notice this extractor only looks for 'theme' field in dataset special 'others' and then 'extras', and doesn't fall back on groups.
protected eu.trentorise.opendata.commons.Dict	extractTitle(CkanDataset dataset, Locale locale)
protected eu.trentorise.opendata.commons.Dict	extractTitle(CkanResource resource, Locale locale)
protected String	extractUri(CkanDataset dataset, String catalogUrl)
protected String	extractUri(CkanResource resource, String catalogUrl, String datasetId)
protected String	formatLanguages(Iterable<Locale> locales) Formats languages list so they can be put into a ckan extras field as string (i.e.
protected String	formatTimestamp(Timestamp timestamp) Formats CKAN timestamp according to ISO 8601.
protected Logger	getLogger() Returns internal logger
protected com.fasterxml.jackson.databind.ObjectMapper	getObjectMapper() Returns internal object mapper
protected static boolean	isTrimmedEmpty(String s) Return true if the provided string is empty after getting trimmed.
protected void	logCantExtract(String clazz, String attribute, Throwable ex)
protected void	logCantFind(String clazz, String attribute)
protected void	logDatasetCantExtract(String attribute, Throwable ex)
protected void	logDatasetCantFind(String attribute)
protected void	logDistribCantExtract(String attribute, Throwable ex)
protected void	logDistribCantFind(String attribute)
eu.trentorise.opendata.traceprov.dcat.DcatDataset	makeDataset(CkanDataset dataset, String catalogUrl, Locale locale) Converts a CkanDataset to a DcatDataset.
eu.trentorise.opendata.traceprov.dcat.DcatDistribution	makeDistribution(CkanResource resource, String catalogURL, String datasetIdOrName, String license, Locale locale) Converts a Ckan resource to a DcatDistribution.
protected void	postProcessDataset(eu.trentorise.opendata.traceprov.dcat.DcatDataset.Builder datasetBuilder, String catalogUrl, Locale locale) Post process the dataset builder after the extractions and prior to creating the immutable DcatDataset object.
protected void	postProcessDistribution(eu.trentorise.opendata.traceprov.dcat.DcatDistribution.Builder distributionBuilder, CkanResource resource, String catalogURL, String datasetId, String license, Locale locale) Post processes the distribution builder after the extractions and prior to creating the immutable DcatDistribution object.
protected void	setLogger(Logger logger) Sets internal logger
protected void	setObjectMapper(com.fasterxml.jackson.databind.ObjectMapper objectMapper) Sets internal object mapper, registering also required modules of traceprov
protected static String	trim(String s) Returns a new string with spaces removed at begin and end.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ISSUED

protected static final String ISSUED

See Also:

Constant Field Values

MODIFIED

protected static final String MODIFIED

See Also:

Constant Field Values

DESCRIPTION

protected static final String DESCRIPTION

See Also:

Constant Field Values

URI_FIELD

protected static final String URI_FIELD

See Also:

Constant Field Values

TITLE

protected static final String TITLE

See Also:

Constant Field Values

Constructor Detail

DcatFactory

public DcatFactory()

Creates a factory with default configuration.

Method Detail

getLogger

protected Logger getLogger()

Returns internal logger

setLogger

protected void setLogger(Logger logger)

Sets internal logger

getObjectMapper

protected com.fasterxml.jackson.databind.ObjectMapper getObjectMapper()

Returns internal object mapper

setObjectMapper

protected void setObjectMapper(com.fasterxml.jackson.databind.ObjectMapper objectMapper)

Sets internal object mapper, registering also required modules of traceprov

formatLanguages

protected String formatLanguages(Iterable<Locale> locales)

Formats languages list so they can be put into a ckan extras field as string (i.e. "[\"ca\", \"en\", \"es\"]")

Throws:

JackanException - on error

extractLanguages

protected List<Locale> extractLanguages(CkanDataset dataset)

i.e. "[\"ca\", \"en\", \"es\"]"

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractFieldAsNonEmptyString

protected String extractFieldAsNonEmptyString(CkanDataset dataset,
                                  String field)

Like extractFieldAsString(CkanDataset, String) but also checks for trimmed non-emptiness.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractFieldAsNonEmptyString

protected String extractFieldAsNonEmptyString(CkanResource resource,
                                  String field)

Like extractFieldAsString(CkanResource, String) but also checks for trimmed non-emptiness.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractFieldAsString

protected String extractFieldAsString(CkanDataset dataset,
                          String field)

Searches a field in CkanDatasetBase.getOthers() and then in CkanDatasetBase.getExtras(). If search fails throws JackanNotFoundException, even if field is found but has null value.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractFieldAsString

protected String extractFieldAsString(CkanResource resource,
                          String field)

Searches a field in CkanResourceBase.getOthers(). If search fails throws JackanNotFoundException, even if field is found but has null value.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractFieldAsObject

protected Object extractFieldAsObject(CkanDataset dataset,
                          String field)

Searches a field in CkanDatasetBase.getOthers() and then in CkanDatasetBase.getExtras(). If search fails throws JackanNotFoundException, even if field is found but has null value.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractField

protected <T> T extractField(CkanDataset dataset,
                 String field,
                 com.fasterxml.jackson.core.type.TypeReference<T> toType)

Tries to extract a string field from a CkanDataset and casts it to target type

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractField

protected <T> T extractField(CkanDataset dataset,
                 String field,
                 Class<T> toClass)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

See Also:

extractField(eu.trentorise.opendata.jackan.model.CkanDataset, java.lang.String, com.fasterxml.jackson.core.type.TypeReference)

formatTimestamp

protected String formatTimestamp(Timestamp timestamp)

Formats CKAN timestamp according to ISO 8601. Differently from CKAN, it adds a 'Z' for clarity.

extractSpatial

protected eu.trentorise.opendata.traceprov.geojson.GeoJson extractSpatial(CkanDataset dataset)

Throws:

JackanNotFoundException - if spatial is not found

JackanException - for other errors.

extractThemes

protected List<eu.trentorise.opendata.traceprov.dcat.SkosConcept> extractThemes(CkanDataset dataset,
                                                                    Locale locale,
                                                                    String catalogUrl)

Notice this extractor only looks for 'theme' field in dataset special 'others' and then 'extras', and doesn't fall back on groups. In case nothing is found, just returns an empty collection.

Parameters:

locale - the locale of the theme names. If unknown pass Locale.ROOT

Throws:

JackanNotFoundException - if needed fields are missing.

JackanException - on generic error

extractUri

protected String extractUri(CkanDataset dataset,
                String catalogUrl)

Parameters:

catalogUrl - i.e. http://dati.trentino.it

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractTemporal

protected eu.trentorise.opendata.commons.PeriodOfTime extractTemporal(CkanDataset dataset)

Returns a string with values trying to respect ISO 8601 format for time intervals: https://en.wikipedia.org/wiki/ISO_8601#Time_intervals

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractIdentifier

protected String extractIdentifier(CkanDataset dataset)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractIssued

protected String extractIssued(CkanDataset dataset)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractModified

protected String extractModified(CkanDataset dataset)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractPublisher

protected eu.trentorise.opendata.traceprov.dcat.FoafAgent extractPublisher(CkanDataset dataset,
                                                               Locale locale)

Notice this extractor will mostly look for special dcat fields in dataset, without resorting to ckan group, organization or maintainer as fallback.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractContactPoint

protected eu.trentorise.opendata.traceprov.dcat.VCard extractContactPoint(CkanDataset dataset)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractKeywords

protected List<String> extractKeywords(CkanDataset dataset)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

trim

protected static String trim(@Nullable
          String s)

Returns a new string with spaces removed at begin and end. If provided string is null returns the empty string.

extractTitle

protected eu.trentorise.opendata.commons.Dict extractTitle(CkanDataset dataset,
                                               Locale locale)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractDescription

protected eu.trentorise.opendata.commons.Dict extractDescription(CkanDataset dataset,
                                                     Locale locale)

Parameters:

locale - if unknown pass Locale.ROOT

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractAccrualPeriodicity

protected String extractAccrualPeriodicity(CkanDataset dataset)

extractLandingPage

protected String extractLandingPage(CkanDataset dataset)

logCantFind

protected void logCantFind(String clazz,
               String attribute)

logCantExtract

protected void logCantExtract(String clazz,
                  String attribute,
                  Throwable ex)

logDatasetCantFind

protected void logDatasetCantFind(String attribute)

logDatasetCantExtract

protected void logDatasetCantExtract(String attribute,
                         Throwable ex)

makeDataset

@Beta
public eu.trentorise.opendata.traceprov.dcat.DcatDataset makeDataset(CkanDataset dataset,
                                                                 String catalogUrl,
                                                                 Locale locale)

Converts a CkanDataset to a DcatDataset. If the dataset contains erroneus fields the converter should just skip them without throwing exceptions.

Parameters:

dataset - must be non null, but it may have missing or null fields.

catalogUrl - non-null catalog url, i.e. "http://dati.trentino.it" or empty one ""

locale - the locale of metadata text. If locale is unknown, use Locale.ROOT. todo write about data locale

postProcessDataset

protected void postProcessDataset(eu.trentorise.opendata.traceprov.dcat.DcatDataset.Builder datasetBuilder,
                      String catalogUrl,
                      Locale locale)

Post process the dataset builder after the extractions and prior to creating the immutable DcatDataset object. Override this method in case you want to perform consistency checks or reset some field.

Parameters:

datasetBuilder -

See Also:

postProcessDistribution(eu.trentorise.opendata.traceprov.dcat.DcatDistribution.Builder, CkanResource, String, String, String, Locale)

logDistribCantFind

protected void logDistribCantFind(String attribute)

logDistribCantExtract

protected void logDistribCantExtract(String attribute,
                         Throwable ex)

extractUri

protected String extractUri(CkanResource resource,
                String catalogUrl,
                String datasetId)

Parameters:

resource -

catalogUrl - i.e. http://dati.trentino.it

datasetId -

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

isTrimmedEmpty

protected static boolean isTrimmedEmpty(@Nullable
                     String s)

Return true if the provided string is empty after getting trimmed.

postProcessDistribution

protected void postProcessDistribution(eu.trentorise.opendata.traceprov.dcat.DcatDistribution.Builder distributionBuilder,
                           CkanResource resource,
                           String catalogURL,
                           String datasetId,
                           String license,
                           Locale locale)

Post processes the distribution builder after the extractions and prior to creating the immutable DcatDistribution object. Override this method in case you want to perform consistency checks or reset some field.

Parameters:

distributionBuilder -

See Also:

postProcessDistribution(eu.trentorise.opendata.traceprov.dcat.DcatDistribution.Builder, CkanResource, String, String, String, Locale), postProcessDataset(eu.trentorise.opendata.traceprov.dcat.DcatDataset.Builder, java.lang.String, java.util.Locale)

extractModified

protected String extractModified(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractIssued

protected String extractIssued(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractAccessUrl

protected String extractAccessUrl(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractDownloadUrl

protected String extractDownloadUrl(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractByteSize

protected int extractByteSize(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractDescription

protected eu.trentorise.opendata.commons.Dict extractDescription(CkanResource dataset,
                                                     Locale locale)

Parameters:

locale - if unknown pass Locale.ROOT

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractFormat

protected String extractFormat(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractLicense

protected String extractLicense(CkanResource resource,
                    String license)

Parameters:

license - value used if resource does not already have a license field. If unknown pass the empty string.

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractMediaType

protected String extractMediaType(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractRights

protected String extractRights(CkanResource resource)

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

extractTitle

protected eu.trentorise.opendata.commons.Dict extractTitle(CkanResource resource,
                                               Locale locale)

Parameters:

locale - if unknown pass Locale.ROOT

Throws:

JackanNotFoundException - when not found

JackanException - on generic error

makeDistribution

@Beta
public eu.trentorise.opendata.traceprov.dcat.DcatDistribution makeDistribution(CkanResource resource,
                                                                           String catalogURL,
                                                                           String datasetIdOrName,
                                                                           String license,
                                                                           Locale locale)

Converts a Ckan resource to a DcatDistribution. If the resource contains erroneus fields the converter should just skip them without throwing exceptions.

Parameters:

resource - must be non null, but it may have missing or null fields.

catalogURL - catalog string, i.e. http://dati.trentino.it

datasetIdOrName - owner dataset alphanumerical id (i.e. fccc07ce-3750-4970-92fd-6a6f432b4466, preferred as stable) or dataset name (less preferred, as names can change over time)

license - A link to the license document under which the distribution is made available. For more info, see ADcatDistribution.getLicense() . If license is unknown, use an empty string.

locale - The language of the distribution. if unknown use Locale.ROOT