about TDB JENA

[Manuel Enrique Puebla Martinez]

Hello:

Is it possible to use some reasoner with RDF data, managed from TDB?

Is it possible to manage an ontology in OWL2 from TDB?

La @universidad_uci es Fidel. Los jóvenes no fallaremos.
#HastaSiempreComandante
#HastalaVictoriaSiempre

[Lorenz B.]

Post by Manuel Enrique Puebla Martinez
Is it possible to use some reasoner with RDF data, managed from TDB?

Yes, see [1]

Post by Manuel Enrique Puebla Martinez
Is it possible to manage an ontology in OWL2 from TDB?

What means "manage"? In general you can load and query any RDF data into
TDB. The serialization of an OWL ontology can be any RDF format.
Obviously, querying OWL constructs via SPARQL can become weird because
an OWL axiom/class expression can consists of more than one RDF triple.

[1] https://jena.apache.org/documentation/tdb/assembler.html

[Manuel Enrique Puebla Martinez]

I consider that I did not know how to explain correctly in my previous email, I repeat the two questions:


1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.

I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.

Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?

I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Best Regards, Manuel Puebla.


----- Mensaje original -----
De: "Lorenz B." <***@informatik.uni-leipzig.de>
Para: ***@jena.apache.org
Enviados: Domingo, 19 de Marzo 2017 3:52:30
Asunto: [MASSMAIL]Re: about TDB JENA

Post by Manuel Enrique Puebla Martinez
Is it possible to use some reasoner with RDF data, managed from TDB?

Yes, see [1]

Post by Manuel Enrique Puebla Martinez
Is it possible to manage an ontology in OWL2 from TDB?

What means "manage"? In general you can load and query any RDF data into
TDB. The serialization of an OWL ontology can be any RDF format.
Obviously, querying OWL constructs via SPARQL can become weird because
an OWL axiom/class expression can consists of more than one RDF triple.

[1] https://jena.apache.org/documentation/tdb/assembler.html

[Dave Reynolds]

Post by Manuel Enrique Puebla Martinez
1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.
I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.
Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?
I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Indeed Jena does not offer fully scalable reasoning, all inference is
done in memory.

That said 13 million assertions is not *that* enormous, the cost of
inference depends on the complexity of the ontology as much its scale.
So 13m triples with some simple domain/range inferences might work in
memory.

TDB storage itself scales just fine and querying does not load all the
data into memory. So if you don't actually need inference, or only need
simple inference that can be usefully expressed as part of the SPARQL
query then you are fine.

Dave

[A. Soroka]

Just a side note; Jena offers SPARQL 1.1, which includes property paths [1]. In some situations, they can be used to do some forms of inference (e.g. some kinds of problems involving subsumption) right in your SPARQL queries.

---
A. Soroka
The University of Virginia Library

[1] https://www.w3.org/TR/sparql11-query/#propertypaths

Post by Manuel Enrique Puebla Martinez
1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.
I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.
Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?
I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Indeed Jena does not offer fully scalable reasoning, all inference is done in memory.
That said 13 million assertions is not *that* enormous, the cost of inference depends on the complexity of the ontology as much its scale. So 13m triples with some simple domain/range inferences might work in memory.
TDB storage itself scales just fine and querying does not load all the data into memory. So if you don't actually need inference, or only need simple inference that can be usefully expressed as part of the SPARQL query then you are fine.
Dave

[Manuel Enrique Puebla Martinez]

I keep going into details, thank you for responding.

Of the 13 million property assertions, almost 80% are assertions of object properties, ie relationships between individuals. In the last ontology I generated automatically, only for one of the municipalities in Cuba, I had 27 763 887 of object properties assertions, 105 054 data property assertions and 8 158 individuals.

The inference I need is basically the following:

1) To know all the individuals that belong to a class directly and indirectly, taking into consideration the equivalence between classes and between individuals.

2) Given an individual (Ind) and an object property (OP), know all individuals related to "Ind", through OP. Considering the following characteristics of OP: symmetry, functional, transitivity, inverse, equivalence.

3) Search the direct and indirect subclasses of a class.

4) Identify all classes equivalent to a class, considering that the equivalence relation is transitive.

5) Identify the set of superclasses of a class.

Could JENA and TDB afford that kind of inference on my big ontologies?

Excuse me, but I'm not a deep connoisseur of the SPARQL language. I have only used it to access data that is explicit on the ontology, similar to SQL in relational databases, I have never used it (nor do I know if it is possible to do so) to infer implicit knowledge.

I put copy to Ignazio Palmisano, an excellent researcher and connoisseur of the framework OWLAPI. With which I have been exchanging on this subject.

Best regards.


----- Mensaje original -----
De: "Dave Reynolds" <***@gmail.com>
Para: ***@jena.apache.org
Enviados: Domingo, 19 de Marzo 2017 13:45:48
Asunto: Re: [MASSMAIL]Re: about TDB JENA

Post by Manuel Enrique Puebla Martinez
1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.
I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.
Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?
I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Indeed Jena does not offer fully scalable reasoning, all inference is
done in memory.

That said 13 million assertions is not *that* enormous, the cost of
inference depends on the complexity of the ontology as much its scale.
So 13m triples with some simple domain/range inferences might work in
memory.

TDB storage itself scales just fine and querying does not load all the
data into memory. So if you don't actually need inference, or only need
simple inference that can be usefully expressed as part of the SPARQL
query then you are fine.

Dave

La @universidad_uci es Fidel. Los jóvenes no fallaremos.
#HastaSiempreComandante
#HastalaVictoriaSiempre

[Lorenz B.]

It totally depends on the reasoning that you want to apply. OWL 2 DL is
not possible via simple rules, but for instance RDFS/OLW Horst and OWL
RL can be doen via rule-based materialization.

Post by Manuel Enrique Puebla Martinez
I keep going into details, thank you for responding.
Of the 13 million property assertions, almost 80% are assertions of object properties, ie relationships between individuals. In the last ontology I generated automatically, only for one of the municipalities in Cuba, I had 27 763 887 of object properties assertions, 105 054 data property assertions and 8 158 individuals.
1) To know all the individuals that belong to a class directly and indirectly, taking into consideration the equivalence between classes and between individuals.

Depends on the reasoning profile and the ontology schema, but might be
covered by SPARQL 1.1 as long as you need only RDFS/OWL RL reasoning.

Post by Manuel Enrique Puebla Martinez
2) Given an individual (Ind) and an object property (OP), know all individuals related to "Ind", through OP. Considering the following characteristics of OP: symmetry, functional, transitivity, inverse, equivalence.
3) Search the direct and indirect subclasses of a class.

SPARQL 1.1 property paths as long as the classes are atomic classes and
not complex class expressions.

Post by Manuel Enrique Puebla Martinez
4) Identify all classes equivalent to a class, considering that the equivalence relation is transitive.
5) Identify the set of superclasses of a class.

SPARQL 1.1 property paths as long as the classes are atomic classes and
not complex class expressions.

Post by Manuel Enrique Puebla Martinez
Could JENA and TDB afford that kind of inference on my big ontologies?
Excuse me, but I'm not a deep connoisseur of the SPARQL language. I have only used it to access data that is explicit on the ontology, similar to SQL in relational databases, I have never used it (nor do I know if it is possible to do so) to infer implicit knowledge.

The approach that people do is either query rewriting w.r.t. the schema
or forward-chaining, i.e. materialization based on a set of inference
rules. For RDFS, OWL Horst and OWL RL this is possible. Materialization
has to be done only once (given that the dataset does not change).

Post by Manuel Enrique Puebla Martinez
I put copy to Ignazio Palmisano, an excellent researcher and connoisseur of the framework OWLAPI. With which I have been exchanging on this subject.
Best regards.
----- Mensaje original -----
Enviados: Domingo, 19 de Marzo 2017 13:45:48
Asunto: Re: [MASSMAIL]Re: about TDB JENA

Post by Manuel Enrique Puebla Martinez
1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.
I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.
Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?
I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Indeed Jena does not offer fully scalable reasoning, all inference is
done in memory.
That said 13 million assertions is not *that* enormous, the cost of
inference depends on the complexity of the ontology as much its scale.
So 13m triples with some simple domain/range inferences might work in
memory.
TDB storage itself scales just fine and querying does not load all the
data into memory. So if you don't actually need inference, or only need
simple inference that can be usefully expressed as part of the SPARQL
query then you are fine.
Dave
#HastaSiempreComandante
#HastalaVictoriaSiempre

[Manuel Enrique Puebla Martinez]

I'm not sure I completely understood your answer. Please confirm my interpretation.

I think I have understood, that the solution is to write rule-based materialization that substitute the reasoners. Apparently since TDB it is possible to execute those rules of inferences on my big ontology, is it?

It seems that rule-based materialization is not applicable to OWL2 ontologies (which do not fit into any of the profiles), is it?

Greetings and thank you very much for your time.


----- Mensaje original -----
De: "Lorenz B." <***@informatik.uni-leipzig.de>
Para: ***@jena.apache.org
Enviados: Lunes, 20 de Marzo 2017 2:24:33
Asunto: Re: [MASSMAIL]Re: about TDB JENA


It totally depends on the reasoning that you want to apply. OWL 2 DL is
not possible via simple rules, but for instance RDFS/OLW Horst and OWL
RL can be doen via rule-based materialization.

Post by Manuel Enrique Puebla Martinez
I keep going into details, thank you for responding.
Of the 13 million property assertions, almost 80% are assertions of object properties, ie relationships between individuals. In the last ontology I generated automatically, only for one of the municipalities in Cuba, I had 27 763 887 of object properties assertions, 105 054 data property assertions and 8 158 individuals.
1) To know all the individuals that belong to a class directly and indirectly, taking into consideration the equivalence between classes and between individuals.

Depends on the reasoning profile and the ontology schema, but might be
covered by SPARQL 1.1 as long as you need only RDFS/OWL RL reasoning.

Post by Manuel Enrique Puebla Martinez
2) Given an individual (Ind) and an object property (OP), know all individuals related to "Ind", through OP. Considering the following characteristics of OP: symmetry, functional, transitivity, inverse, equivalence.
3) Search the direct and indirect subclasses of a class.

SPARQL 1.1 property paths as long as the classes are atomic classes and
not complex class expressions.

Post by Manuel Enrique Puebla Martinez
4) Identify all classes equivalent to a class, considering that the equivalence relation is transitive.
5) Identify the set of superclasses of a class.

SPARQL 1.1 property paths as long as the classes are atomic classes and
not complex class expressions.

Post by Manuel Enrique Puebla Martinez
Could JENA and TDB afford that kind of inference on my big ontologies?
Excuse me, but I'm not a deep connoisseur of the SPARQL language. I have only used it to access data that is explicit on the ontology, similar to SQL in relational databases, I have never used it (nor do I know if it is possible to do so) to infer implicit knowledge.

The approach that people do is either query rewriting w.r.t. the schema
or forward-chaining, i.e. materialization based on a set of inference
rules. For RDFS, OWL Horst and OWL RL this is possible. Materialization
has to be done only once (given that the dataset does not change).

Post by Manuel Enrique Puebla Martinez
I put copy to Ignazio Palmisano, an excellent researcher and connoisseur of the framework OWLAPI. With which I have been exchanging on this subject.
Best regards.
----- Mensaje original -----
Enviados: Domingo, 19 de Marzo 2017 13:45:48
Asunto: Re: [MASSMAIL]Re: about TDB JENA

Post by Manuel Enrique Puebla Martinez
1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.
I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.
Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?
I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Indeed Jena does not offer fully scalable reasoning, all inference is
done in memory.
That said 13 million assertions is not *that* enormous, the cost of
inference depends on the complexity of the ontology as much its scale.
So 13m triples with some simple domain/range inferences might work in
memory.
TDB storage itself scales just fine and querying does not load all the
data into memory. So if you don't actually need inference, or only need
simple inference that can be usefully expressed as part of the SPARQL
query then you are fine.
Dave
#HastaSiempreComandante
#HastalaVictoriaSiempre

[A. Soroka]

The OWL 2 RL profile is aimed at applications that require scalable reasoning without sacrificing too much expressive power. It is designed to accommodate both OWL 2 applications that can trade the full expressivity of the language for efficiency, and RDF(S) applications that need some added expressivity from OWL 2. This is achieved by defining a syntactic subset of OWL 2 which is amenable to implementation using rule-based technologies (see Section 4.2), and presenting a partial axiomatization of the OWL 2 RDF-Based Semantics in the form of first-order implications that can be used as the basis for such an implementation (see Section 4.3).

See https://www.w3.org/TR/owl2-profiles/#OWL_2_RL

But if you mean to say that your ontology is known not to fit any of the OWL 2 profiles, then you can still use rules (and SPARQL) for some of the problems you may wish to solve, but not others. For example, as Lorenz (and I) remarked, if your classes are atomic you can use SPARQL property paths to solve subsumption problems.

You may not be able to throw all of your data and problems into a single "inference machine", but you may very well be able to solve most or all of your problems using different techniques.

---
A. Soroka
The University of Virginia Library

I'm not sure I completely understood your answer. Please confirm my interpretation.
I think I have understood, that the solution is to write rule-based materialization that substitute the reasoners. Apparently since TDB it is possible to execute those rules of inferences on my big ontology, is it?
It seems that rule-based materialization is not applicable to OWL2 ontologies (which do not fit into any of the profiles), is it?
Greetings and thank you very much for your time.
----- Mensaje original -----
Enviados: Lunes, 20 de Marzo 2017 2:24:33
Asunto: Re: [MASSMAIL]Re: about TDB JENA
It totally depends on the reasoning that you want to apply. OWL 2 DL is
not possible via simple rules, but for instance RDFS/OLW Horst and OWL
RL can be doen via rule-based materialization.

Post by Manuel Enrique Puebla Martinez
I keep going into details, thank you for responding.
Of the 13 million property assertions, almost 80% are assertions of object properties, ie relationships between individuals. In the last ontology I generated automatically, only for one of the municipalities in Cuba, I had 27 763 887 of object properties assertions, 105 054 data property assertions and 8 158 individuals.
1) To know all the individuals that belong to a class directly and indirectly, taking into consideration the equivalence between classes and between individuals.

Depends on the reasoning profile and the ontology schema, but might be
covered by SPARQL 1.1 as long as you need only RDFS/OWL RL reasoning.

Post by Manuel Enrique Puebla Martinez
2) Given an individual (Ind) and an object property (OP), know all individuals related to "Ind", through OP. Considering the following characteristics of OP: symmetry, functional, transitivity, inverse, equivalence.
3) Search the direct and indirect subclasses of a class.

SPARQL 1.1 property paths as long as the classes are atomic classes and
not complex class expressions.

Post by Manuel Enrique Puebla Martinez
4) Identify all classes equivalent to a class, considering that the equivalence relation is transitive.
5) Identify the set of superclasses of a class.

SPARQL 1.1 property paths as long as the classes are atomic classes and
not complex class expressions.

Post by Manuel Enrique Puebla Martinez
Could JENA and TDB afford that kind of inference on my big ontologies?
Excuse me, but I'm not a deep connoisseur of the SPARQL language. I have only used it to access data that is explicit on the ontology, similar to SQL in relational databases, I have never used it (nor do I know if it is possible to do so) to infer implicit knowledge.

The approach that people do is either query rewriting w.r.t. the schema
or forward-chaining, i.e. materialization based on a set of inference
rules. For RDFS, OWL Horst and OWL RL this is possible. Materialization
has to be done only once (given that the dataset does not change).

Post by Manuel Enrique Puebla Martinez
I put copy to Ignazio Palmisano, an excellent researcher and connoisseur of the framework OWLAPI. With which I have been exchanging on this subject.
Best regards.
----- Mensaje original -----
Enviados: Domingo, 19 de Marzo 2017 13:45:48
Asunto: Re: [MASSMAIL]Re: about TDB JENA

Post by Manuel Enrique Puebla Martinez
1) I read the page https://jena.apache.org/documentation/tdb/assembler.html, I do not think it is what I need.
I work with large OWL2 ontologies from the OWLAPI framework, generated automatically. With thousands of individuals and more than 13 million property assertions (data and objects). As one may assume, one of the limitations I have is that OWLAPI itself can not manage these large ontologies, that is, because OWLAPI loads the whole owl file into RAM. Not to dream that some classical reasoner (Pellet, Hermit, etc.) can infer new knowledge about these great ontologies.
Once explained the problem I have, comes the question: Does JENA solve this test ?, ie with JENA and TDB I can generate my great ontologies in OWL2 ?, With JENA and TDB I can use a reasoner to infer new implicit knowledge (unstated) on my big ontologies?
I do not think JENA will be able to solve this problem, it would be a pleasant surprise for me. Unfortunately so far I had not read about TDB and the potentialities of JENA in external memory.

Indeed Jena does not offer fully scalable reasoning, all inference is
done in memory.
That said 13 million assertions is not *that* enormous, the cost of
inference depends on the complexity of the ontology as much its scale.
So 13m triples with some simple domain/range inferences might work in
memory.
TDB storage itself scales just fine and querying does not load all the
data into memory. So if you don't actually need inference, or only need
simple inference that can be usefully expressed as part of the SPARQL
query then you are fine.
Dave
#HastaSiempreComandante
#HastalaVictoriaSiempre

--
Lorenz Bühmann
AKSW group, University of Leipzig
Group: http://aksw.org - semantic web research center
#HastaSiempreComandante
#HastalaVictoriaSiempre