de.gwdg.metadataqa.api.calculator

Class CalculatorFacade

java.lang.Object

de.gwdg.metadataqa.api.calculator.CalculatorFacade

All Implemented Interfaces:

Serializable

public class CalculatorFacade
extends Object
implements Serializable

The central entry point of the application. It provides a facade to the subsystems.

Author:

Péter Király

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected PathCache<? extends XmlFieldInstance>	cache
protected List<Calculator>	calculators The list of registered calculator objects.
protected boolean	checkSkippableCollections
protected boolean	collectTfIdfTerms Flag whether or not collect TF-IDF terms in uniqueness measurement (default: false).
protected CompletenessCalculator	completenessCalculator The completeness calculator.
protected boolean	completenessCollectFields Flag whether or not run missing/empty/existing field collection in completeness (default: false).
protected boolean	completenessMeasurementEnabled Flag whether or not run the completeness measurement (default: true).
protected CompressionLevel	compressionLevel
protected CsvReader	csvReader
protected boolean	fieldCardinalityMeasurementEnabled Flag whether or not run the field cardinality measurement (default: true).
protected boolean	fieldExistenceMeasurementEnabled Flag whether or not run the field existence measurement (default: true).
protected FieldExtractor	fieldExtractor The field extractor object.
protected boolean	fieldExtractorEnabled Flag whether or not the field extractor is enabled (default: false).
protected Format	format
protected LanguageCalculator	languageCalculator The language detector.
protected boolean	languageMeasurementEnabled Flag whether or not run the language detector (default: false).
protected MultilingualitySaturationCalculator	multilingualSaturationCalculator The language detector.
protected boolean	multilingualSaturationMeasurementEnabled Flag whether or not run the multilingual saturation measurement (default: false).
protected boolean	problemCatalogMeasurementEnabled Flag whether or not run the problem catalog (default: false).
protected boolean	ruleCatalogMeasurementEnabled Flag whether or not run the rule catalog (default: false).
protected boolean	saturationExtendedResult
protected Schema	schema
protected SolrClient	solrClient
protected SolrConfiguration	solrConfiguration
protected TfIdfCalculator	tfidfCalculator The TF-IDF calculator.
protected boolean	tfIdfMeasurementEnabled Flag whether or not run the uniqueness measurement (default: false).
protected boolean	uniquenessMeasurementEnabled

Constructor Summary

Constructors

Constructor and Description
CalculatorFacade() Create calculator facade with the default configuration.
CalculatorFacade(boolean runFieldExistence, boolean runFieldCardinality, boolean runCompleteness, boolean runTfIdf, boolean runProblemCatalog) Create calculator facade with configuration.

Method Summary

Modifier and Type	Method and Description
boolean	collectTfIdfTerms() Returns the flag whether the measurement should collect each individual terms with their Term Ferquency and Invers Document Frequency scores.
CalculatorFacade	collectTfIdfTerms(boolean collectTfIdfTerms) Sets the flag whether the measurement should collect each individual terms with their Term Ferquency and Invers Document Frequency scores.
boolean	completenessCollectFields() Get the completenessCollectFields flag.
CalculatorFacade	completenessCollectFields(boolean completenessCollectFields) The completeness calculation will collect empty, existent and missing fields.
protected void	conditionalConfiguration()
void	configure() Run the configuration based on the previously set flags.
CalculatorFacade	configureSolr(String solrHost, String solrPort, String solrPath)
CalculatorFacade	disableCompletenessMeasurement()
CalculatorFacade	disableFieldCardinalityMeasurement()
CalculatorFacade	disableFieldExistenceMeasurement()
CalculatorFacade	disableFieldExtractor()
CalculatorFacade	disableLanguageMeasurement()
CalculatorFacade	disableMultilingualSaturationMeasurement()
CalculatorFacade	disableProblemCatalogMeasurement()
CalculatorFacade	disableRuleCatalogMeasurement()
CalculatorFacade	disableTfIdfMeasurement()
CalculatorFacade	disableUniquenessMeasurement()
CalculatorFacade	enableCompletenessMeasurement()
CalculatorFacade	enableCompletenessMeasurement(boolean runCompleteness) Sets the flag whether or not run the completeness measurement.
CalculatorFacade	enableFieldCardinalityMeasurement()
CalculatorFacade	enableFieldCardinalityMeasurement(boolean runFieldCardinality) configure to run the cardinality measurement.
CalculatorFacade	enableFieldExistenceMeasurement()
CalculatorFacade	enableFieldExistenceMeasurement(boolean runFieldExistence) Sets whether or not to run the field existence measurement.
CalculatorFacade	enableFieldExtractor()
CalculatorFacade	enableFieldExtractor(boolean flag)
CalculatorFacade	enableLanguageMeasurement()
CalculatorFacade	enableLanguageMeasurement(boolean runLanguage) Configure whether or not run the language detector.
CalculatorFacade	enableMultilingualSaturationMeasurement()
CalculatorFacade	enableMultilingualSaturationMeasurement(boolean runMultilingualSaturation) Configure whether or not run the language detector.
CalculatorFacade	enableProblemCatalogMeasurement()
CalculatorFacade	enableProblemCatalogMeasurement(boolean runProblemCatalog) Configure to run the problem catalog measurement.
CalculatorFacade	enableRuleCatalogMeasurement()
CalculatorFacade	enableRuleCatalogMeasurement(boolean run) Configure to run the problem catalog measurement.
CalculatorFacade	enableTfIdfMeasurement()
CalculatorFacade	enableTfIdfMeasurement(boolean runTfIdf) Configure whether or not run the uniqueness measurement.
CalculatorFacade	enableUniquenessMeasurement()
CalculatorFacade	enableUniquenessMeasurement(boolean uniquenessMeasurementEnabled) Flag to enable uniqueness measurement.
PathCache<? extends XmlFieldInstance>	getCache()
List<Calculator>	getCalculators() Return the list of all registered calculators.
CompressionLevel	getCompressionLevel()
String	getCsv(boolean withLabels, CompressionLevel compressionLevel)
List<String>	getEmptyFields() Returns the list of empty fields.
List<String>	getExistingFields() Returns the list of existing fields.
List<String>	getHeader()
Map<String,Map<String,? extends Object>>	getLabelledResults()
List<String>	getMissingFields() Returns the list of missing fields.
Map<String,Object>	getResults() Returns the result map.
Schema	getSchema()
Map<String,List<TfIdf>>	getTermsCollection() Returns the TF-IDF term map.
boolean	isCheckSkippableCollections()
boolean	isCompletenessMeasurementEnabled() Returns the flag whether or not run the completeness measurement.
boolean	isFieldCardinalityMeasurementEnabled() Returns whether or not to run cardinality measurement.
boolean	isFieldExistenceMeasurementEnabled() Returns whether or not to run the field existence measurement.
boolean	isFieldExtractorEnabled()
boolean	isLanguageMeasurementEnabled() Returns the flag whether or not run the language detector.
boolean	isMultilingualSaturationMeasurementEnabled() Returns the flag whether or not run the language detector.
boolean	isProblemCatalogMeasurementEnabled() Gets flag whether to run the problem catalog measurement.
boolean	isRuleCatalogMeasurementEnabled() Gets flag whether to run the rule catalog measurement.
boolean	isSaturationExtendedResult()
boolean	isTfIdfMeasurementEnabled() Returns whether or not run the uniqueness measurement.
boolean	isUniquenessMeasurementEnabled() Is uniqueness measurement enabled?
String	measure(String jsonRecord) Run the measurements with each Calculator then returns the result as CSV.
List<String>	measureAsList(String jsonRecord)
Map<String,Object>	measureAsMap(String jsonRecord)
protected <T extends XmlFieldInstance> Object	measureWithGenerics(String content) The generic version of measure.
protected <T extends XmlFieldInstance> Object	measureWithGenerics(String content, Class outputClass)
CalculatorFacade	setCheckSkippableCollections(boolean checkSkippableCollections)
CalculatorFacade	setCompressionLevel(CompressionLevel compressionLevel)
CalculatorFacade	setCsvReader(CsvReader csvReader)
CalculatorFacade	setSaturationExtendedResult(boolean saturationExtendedResult)
CalculatorFacade	setSchema(Schema schema)
CalculatorFacade	setSolrClient(SolrClient solrClient)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fieldExtractorEnabled

protected boolean fieldExtractorEnabled

Flag whether or not the field extractor is enabled (default: false).

fieldExistenceMeasurementEnabled

protected boolean fieldExistenceMeasurementEnabled

Flag whether or not run the field existence measurement (default: true).

fieldCardinalityMeasurementEnabled

protected boolean fieldCardinalityMeasurementEnabled

Flag whether or not run the field cardinality measurement (default: true).

completenessMeasurementEnabled

protected boolean completenessMeasurementEnabled

Flag whether or not run the completeness measurement (default: true).

tfIdfMeasurementEnabled

protected boolean tfIdfMeasurementEnabled

Flag whether or not run the uniqueness measurement (default: false).

problemCatalogMeasurementEnabled

protected boolean problemCatalogMeasurementEnabled

Flag whether or not run the problem catalog (default: false).

ruleCatalogMeasurementEnabled

protected boolean ruleCatalogMeasurementEnabled

Flag whether or not run the rule catalog (default: false).

languageMeasurementEnabled

protected boolean languageMeasurementEnabled

Flag whether or not run the language detector (default: false).

multilingualSaturationMeasurementEnabled

protected boolean multilingualSaturationMeasurementEnabled

Flag whether or not run the multilingual saturation measurement (default: false).

collectTfIdfTerms

protected boolean collectTfIdfTerms

Flag whether or not collect TF-IDF terms in uniqueness measurement (default: false).

completenessCollectFields

protected boolean completenessCollectFields

Flag whether or not run missing/empty/existing field collection in completeness (default: false).

saturationExtendedResult

protected boolean saturationExtendedResult

checkSkippableCollections

protected boolean checkSkippableCollections

uniquenessMeasurementEnabled

protected boolean uniquenessMeasurementEnabled

compressionLevel

protected CompressionLevel compressionLevel

solrClient

protected SolrClient solrClient

solrConfiguration

protected SolrConfiguration solrConfiguration

calculators

protected List<Calculator> calculators

The list of registered calculator objects. Those will do the measurements

fieldExtractor

protected FieldExtractor fieldExtractor

The field extractor object.

completenessCalculator

protected CompletenessCalculator completenessCalculator

The completeness calculator.

tfidfCalculator

protected TfIdfCalculator tfidfCalculator

The TF-IDF calculator.

languageCalculator

protected LanguageCalculator languageCalculator

The language detector.

multilingualSaturationCalculator

protected MultilingualitySaturationCalculator multilingualSaturationCalculator

The language detector.

format

protected Format format

cache

protected PathCache<? extends XmlFieldInstance> cache

schema

protected Schema schema

csvReader

protected CsvReader csvReader

Constructor Detail

CalculatorFacade

public CalculatorFacade()

Create calculator facade with the default configuration.

CalculatorFacade

public CalculatorFacade(boolean runFieldExistence,
                        boolean runFieldCardinality,
                        boolean runCompleteness,
                        boolean runTfIdf,
                        boolean runProblemCatalog)

Create calculator facade with configuration.

Parameters:

runFieldExistence - Flag whether or not run the field existence measurement

runFieldCardinality - Flag whether or not run the field cardinality measurement

runCompleteness - Flag whether or not run the completeness measurement

runTfIdf - Flag whether or not run the uniqueness measurement

runProblemCatalog - Flag whether or not run the problem catalog

Method Detail

conditionalConfiguration

protected void conditionalConfiguration()

configure

public void configure()

Run the configuration based on the previously set flags.

measure

public String measure(String jsonRecord)
               throws com.jayway.jsonpath.InvalidJsonException

Run the measurements with each Calculator then returns the result as CSV.

Parameters:

jsonRecord - The JSON record string

Returns:

The result of measurements as a CSV string

Throws:

com.jayway.jsonpath.InvalidJsonException - Invalid Json exception

measureAsList

public List<String> measureAsList(String jsonRecord)
                           throws com.jayway.jsonpath.InvalidJsonException

Throws:

com.jayway.jsonpath.InvalidJsonException

measureAsMap

public Map<String,Object> measureAsMap(String jsonRecord)
                                throws com.jayway.jsonpath.InvalidJsonException

Throws:

com.jayway.jsonpath.InvalidJsonException

measureWithGenerics

protected <T extends XmlFieldInstance> Object measureWithGenerics(String content)
                                                           throws com.jayway.jsonpath.InvalidJsonException

The generic version of measure. The class to pass should define the individual field's representations, so it is bound to the schema the record in. The class should be the extension of XmlFieldInstance.

Type Parameters:

T - A class defining the internal representation of a field. It should be an extension of XmlFieldInstance

Parameters:

content - The JSON record

Returns:

The result of measurements as a CSV string

Throws:

com.jayway.jsonpath.InvalidJsonException

measureWithGenerics

protected <T extends XmlFieldInstance> Object measureWithGenerics(String content,
                                                                  Class outputClass)
                                                           throws com.jayway.jsonpath.InvalidJsonException

Throws:

com.jayway.jsonpath.InvalidJsonException

enableFieldExtractor

public CalculatorFacade enableFieldExtractor()

disableFieldExtractor

public CalculatorFacade disableFieldExtractor()

enableFieldExtractor

public CalculatorFacade enableFieldExtractor(boolean flag)

isFieldExtractorEnabled

public boolean isFieldExtractorEnabled()

isFieldExistenceMeasurementEnabled

public boolean isFieldExistenceMeasurementEnabled()

Returns whether or not to run the field existence measurement.

Returns:

field existence measurement flag

enableFieldExistenceMeasurement

public CalculatorFacade enableFieldExistenceMeasurement()

disableFieldExistenceMeasurement

public CalculatorFacade disableFieldExistenceMeasurement()

enableFieldExistenceMeasurement

public CalculatorFacade enableFieldExistenceMeasurement(boolean runFieldExistence)

Sets whether or not to run the field existence measurement.

Parameters:

runFieldExistence - field existence measurement flag

Returns:

isFieldCardinalityMeasurementEnabled

public boolean isFieldCardinalityMeasurementEnabled()

Returns whether or not to run cardinality measurement.

Returns:

Flag to run cardinality measurement

enableFieldCardinalityMeasurement

public CalculatorFacade enableFieldCardinalityMeasurement()

disableFieldCardinalityMeasurement

public CalculatorFacade disableFieldCardinalityMeasurement()

enableFieldCardinalityMeasurement

public CalculatorFacade enableFieldCardinalityMeasurement(boolean runFieldCardinality)

configure to run the cardinality measurement.

Parameters:

runFieldCardinality - cardinality measurement flag

Returns:

isCompletenessMeasurementEnabled

public boolean isCompletenessMeasurementEnabled()

Returns the flag whether or not run the completeness measurement.

Returns:

Flag whether or not run the completeness measurement

enableCompletenessMeasurement

public CalculatorFacade enableCompletenessMeasurement()

disableCompletenessMeasurement

public CalculatorFacade disableCompletenessMeasurement()

enableCompletenessMeasurement

public CalculatorFacade enableCompletenessMeasurement(boolean runCompleteness)

Sets the flag whether or not run the completeness measurement.

Parameters:

runCompleteness - flag whether or not run the completeness measurement

Returns:

isLanguageMeasurementEnabled

public boolean isLanguageMeasurementEnabled()

Returns the flag whether or not run the language detector.

Returns:

language detector flag

enableLanguageMeasurement

public CalculatorFacade enableLanguageMeasurement()

disableLanguageMeasurement

public CalculatorFacade disableLanguageMeasurement()

enableLanguageMeasurement

public CalculatorFacade enableLanguageMeasurement(boolean runLanguage)

Configure whether or not run the language detector.

Parameters:

runLanguage -

Returns:

isMultilingualSaturationMeasurementEnabled

public boolean isMultilingualSaturationMeasurementEnabled()

Returns the flag whether or not run the language detector.

Returns:

language detector flag

enableMultilingualSaturationMeasurement

public CalculatorFacade enableMultilingualSaturationMeasurement()

disableMultilingualSaturationMeasurement

public CalculatorFacade disableMultilingualSaturationMeasurement()

enableMultilingualSaturationMeasurement

public CalculatorFacade enableMultilingualSaturationMeasurement(boolean runMultilingualSaturation)

Configure whether or not run the language detector.

Parameters:

runMultilingualSaturation -

Returns:

isTfIdfMeasurementEnabled

public boolean isTfIdfMeasurementEnabled()

Returns whether or not run the uniqueness measurement.

Returns:

uniqueness measurement flag

enableTfIdfMeasurement

public CalculatorFacade enableTfIdfMeasurement()

disableTfIdfMeasurement

public CalculatorFacade disableTfIdfMeasurement()

enableTfIdfMeasurement

public CalculatorFacade enableTfIdfMeasurement(boolean runTfIdf)

Configure whether or not run the uniqueness measurement.

Parameters:

runTfIdf - uniqueness measurement flag

Returns:

isProblemCatalogMeasurementEnabled

public boolean isProblemCatalogMeasurementEnabled()

Gets flag whether to run the problem catalog measurement.

Returns:

problem catalog measurement flag

enableProblemCatalogMeasurement

public CalculatorFacade enableProblemCatalogMeasurement()

disableProblemCatalogMeasurement

public CalculatorFacade disableProblemCatalogMeasurement()

enableProblemCatalogMeasurement

public CalculatorFacade enableProblemCatalogMeasurement(boolean runProblemCatalog)

Configure to run the problem catalog measurement.

Parameters:

runProblemCatalog - problem catalog measurement flag

Returns:

isRuleCatalogMeasurementEnabled

public boolean isRuleCatalogMeasurementEnabled()

Gets flag whether to run the rule catalog measurement.

Returns:

problem catalog measurement flag

enableRuleCatalogMeasurement

public CalculatorFacade enableRuleCatalogMeasurement()

disableRuleCatalogMeasurement

public CalculatorFacade disableRuleCatalogMeasurement()

enableRuleCatalogMeasurement

public CalculatorFacade enableRuleCatalogMeasurement(boolean run)

Configure to run the problem catalog measurement.

Parameters:

run - problem catalog measurement flag

Returns:

isUniquenessMeasurementEnabled

public boolean isUniquenessMeasurementEnabled()

Is uniqueness measurement enabled?

Returns:

uniqueness measurement flag

enableUniquenessMeasurement

public CalculatorFacade enableUniquenessMeasurement()

disableUniquenessMeasurement

public CalculatorFacade disableUniquenessMeasurement()

enableUniquenessMeasurement

public CalculatorFacade enableUniquenessMeasurement(boolean uniquenessMeasurementEnabled)

Flag to enable uniqueness measurement.

Parameters:

uniquenessMeasurementEnabled - The flag

Returns:

getCalculators

public List<Calculator> getCalculators()

Return the list of all registered calculators.

Returns:

The calculators

collectTfIdfTerms

public boolean collectTfIdfTerms()

Returns the flag whether the measurement should collect each individual terms with their Term Ferquency and Invers Document Frequency scores.

Returns:

The TF-IDF collector flag

collectTfIdfTerms

public CalculatorFacade collectTfIdfTerms(boolean collectTfIdfTerms)

Sets the flag whether the measurement should collect each individual terms with their Term Ferquency and Invers Document Frequency scores.

Parameters:

collectTfIdfTerms - The TF-IDF collector flag

Returns:

completenessCollectFields

public boolean completenessCollectFields()

Get the completenessCollectFields flag.

Returns:

completenessCollectFields flag

completenessCollectFields

public CalculatorFacade completenessCollectFields(boolean completenessCollectFields)

The completeness calculation will collect empty, existent and missing fields.

Parameters:

completenessCollectFields - The completenessCollectFields flag

Returns:

getExistingFields

public List<String> getExistingFields()

Returns the list of existing fields.

Returns:

The list of existing fields

getEmptyFields

public List<String> getEmptyFields()

Returns the list of empty fields.

Returns:

The list of empty fields

getMissingFields

public List<String> getMissingFields()

Returns the list of missing fields.

Returns:

The list of missing fields

getTermsCollection

public Map<String,List<TfIdf>> getTermsCollection()

Returns the TF-IDF term map. The keys are the field names, the values are lists of TfIdf objects.

Returns:

TF-IDF term list

See Also:

TfIdfCalculator.getTermsCollection()

getResults

public Map<String,Object> getResults()

Returns the result map.

Returns:

The result map

See Also:

Counters.getResults()

getLabelledResults

public Map<String,Map<String,? extends Object>> getLabelledResults()

getCsv

public String getCsv(boolean withLabels,
                     CompressionLevel compressionLevel)

configureSolr

public CalculatorFacade configureSolr(String solrHost,
                                      String solrPort,
                                      String solrPath)

getHeader

public List<String> getHeader()

isSaturationExtendedResult

public boolean isSaturationExtendedResult()

setSaturationExtendedResult

public CalculatorFacade setSaturationExtendedResult(boolean saturationExtendedResult)

getCompressionLevel

public CompressionLevel getCompressionLevel()

setCompressionLevel

public CalculatorFacade setCompressionLevel(CompressionLevel compressionLevel)

getCache

public PathCache<? extends XmlFieldInstance> getCache()

isCheckSkippableCollections

public boolean isCheckSkippableCollections()

setCheckSkippableCollections

public CalculatorFacade setCheckSkippableCollections(boolean checkSkippableCollections)

getSchema

public Schema getSchema()

setSchema

public CalculatorFacade setSchema(Schema schema)

setSolrClient

public CalculatorFacade setSolrClient(SolrClient solrClient)

setCsvReader

public CalculatorFacade setCsvReader(CsvReader csvReader)