Discussion:
many named graphs
Chris Jones
2016-07-04 23:21:07 UTC
Permalink
Hi All,

How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.

I'm writing a system where I'm considering giving each remote client its
own named graph where it uploads data. I'm further considering keeping
historical data, so the client would get a new named graph each time it
sent its data. I'm curious if this is a bad idea.

Chris
Daniel Hernández
2016-07-05 03:38:28 UTC
Permalink
Hi Chris,

It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf

Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Al Shapiro
2016-07-06 03:58:26 UTC
Permalink
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al 








On Monday, July 4, 2016 8:38 PM, Daniel Hernández <***@degu.cl> wrote:


Hi Chris,

It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf

Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Daniel Hernández
2016-07-06 04:55:35 UTC
Permalink
Hi Al,

Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.

Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Al Shapiro
2016-07-06 05:10:36 UTC
Permalink
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al


 

On Tuesday, July 5, 2016 9:55 PM, Daniel Hernández <***@degu.cl> wrote:


Hi Al,

Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.

Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
 
 
  Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
   
Daniel Hernández
2016-07-06 13:45:34 UTC
Permalink
I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.

Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Al Shapiro
2016-07-07 02:14:27 UTC
Permalink
Daniel,
I want to create a Named Graph "X", based on User-Id, the data for each named Graph is created on the fly, when a Match occurs against the Default Graph (which has already been loaded to the Jena TDB)...
I need to know how to generate the Named Graph based on IRI?
Do I create a NameSpace for the IRI?
Please give me an example...
Thank you,
Al 

On Wednesday, July 6, 2016 6:45 AM, Daniel Hernández <***@degu.cl> wrote:


I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.

Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
   
 
  Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
   
   
    Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
     
   
A. Soroka
2016-07-07 13:24:04 UTC
Permalink
What is the syntactical form of your user id values?

---
A. Soroka
The University of Virginia Library
Post by Al Shapiro
Daniel,
I want to create a Named Graph "X", based on User-Id, the data for each named Graph is created on the fly, when a Match occurs against the Default Graph (which has already been loaded to the Jena TDB)...
I need to know how to generate the Named Graph based on IRI?
Do I create a NameSpace for the IRI?
Please give me an example...
Thank you,
Al
I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.
Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Al Shapiro
2016-07-08 03:45:04 UTC
Permalink
A. Soroka,

The syntactical form of my user id values, is a variable containing "U0000001" through "U9999999" - the user id is assigned when an end-user first logs on to the system, with their e-mail and password - as long as the account is valid, the end-user retains the same user id  assigned to it when they first logged on (registered)...

I still need to know how to generate the Named Graph based on IRI, etc?

Thank you,
Al

On Thursday, July 7, 2016 6:25 AM, A. Soroka <***@virginia.edu> wrote:


What is the syntactical form of your user id values?

---
A. Soroka
The University of Virginia Library
Post by Al Shapiro
Daniel,
I want to create a Named Graph "X", based on User-Id, the data for each named Graph is created on the fly, when a Match occurs against the Default Graph (which has already been loaded to the Jena TDB)...
I need to know how to generate the Named Graph based on IRI?
Do I create a NameSpace for the IRI?
Please give me an example...
Thank you,
Al 
I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.
Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
   
 
  Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
   
   
    Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
     
   
Lorenz B.
2016-07-08 07:26:54 UTC
Permalink
Hello Al,

I do not understand why it is so difficult for you to generate some IRIs
for a users that have an ID. Just use your own namespace and append the
ID. And generating graphs depends on the triple store that you use. In
Jena for instance, you could use some methods of the Dataset [1], like
addNamedModel()

[1]
https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html

Kind regards,
Lorenz
Post by Al Shapiro
A. Soroka,
The syntactical form of my user id values, is a variable containing "U0000001" through "U9999999" - the user id is assigned when an end-user first logs on to the system, with their e-mail and password - as long as the account is valid, the end-user retains the same user id assigned to it when they first logged on (registered)...
I still need to know how to generate the Named Graph based on IRI, etc?
Thank you,
Al
What is the syntactical form of your user id values?
---
A. Soroka
The University of Virginia Library
Post by Al Shapiro
Daniel,
I want to create a Named Graph "X", based on User-Id, the data for each named Graph is created on the fly, when a Match occurs against the Default Graph (which has already been loaded to the Jena TDB)...
I need to know how to generate the Named Graph based on IRI?
Do I create a NameSpace for the IRI?
Please give me an example...
Thank you,
Al
I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.
Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
--
Lorenz BÃŒhmann
AKSW group, University of Leipzig
Group: http://aksw.org - semantic web research center
Andy Seaborne
2016-07-08 07:55:39 UTC
Permalink
You can prepend a string to make it a URI.

http:/example/U0000001
http:/example/user?id=U0000001


or even don't do that but allocate a unique for all time URI for this
user instance and

<urn:uuid:f2504c02-2b2b-11b2-807f-024225278a43>
user:id "U0000001" ;
foaf:name "Al Shapiro" ;
foaf:birthday "2016-07-08"^^xsd:dateTime .

and separately hold in a safe a table:

urn:uuid:f2504c02-2b2b-11b2-807f-024225278a43 PASSWORD

The user never see the URN.

Storing the password separately is for security.

URNs can be uniquely generated safely with no practical risk of
collision across multiple machines. SPARQL even has support for it.

Andy
Post by Lorenz B.
Hello Al,
I do not understand why it is so difficult for you to generate some IRIs
for a users that have an ID. Just use your own namespace and append the
ID. And generating graphs depends on the triple store that you use. In
Jena for instance, you could use some methods of the Dataset [1], like
addNamedModel()
[1]
https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html
Kind regards,
Lorenz
Post by Al Shapiro
A. Soroka,
The syntactical form of my user id values, is a variable containing "U0000001" through "U9999999" - the user id is assigned when an end-user first logs on to the system, with their e-mail and password - as long as the account is valid, the end-user retains the same user id assigned to it when they first logged on (registered)...
I still need to know how to generate the Named Graph based on IRI, etc?
Thank you,
Al
What is the syntactical form of your user id values?
---
A. Soroka
The University of Virginia Library
Post by Al Shapiro
Daniel,
I want to create a Named Graph "X", based on User-Id, the data for each named Graph is created on the fly, when a Match occurs against the Default Graph (which has already been loaded to the Jena TDB)...
I need to know how to generate the Named Graph based on IRI?
Do I create a NameSpace for the IRI?
Please give me an example...
Thank you,
Al
I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.
Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Al Shapiro
2016-07-09 03:00:47 UTC
Permalink
Hi Guys!
I guess the real answer is that the Named Graph / Uesr-ID should be generated via Jena and then it can be referenced (used) in Sparql via wrapper creation and usage programming...
I will also review the use of URN's in Sparql...
Thank you for your help...
Al






On Friday, July 8, 2016 12:55 AM, Andy Seaborne <***@apache.org> wrote:


You can prepend a string to make it a URI.

http:/example/U0000001
http:/example/user?id=U0000001


or even don't do that but allocate a unique for all time URI for this
user instance and

<urn:uuid:f2504c02-2b2b-11b2-807f-024225278a43>
    user:id "U0000001" ;
    foaf:name "Al Shapiro" ;
    foaf:birthday "2016-07-08"^^xsd:dateTime .

and separately hold in a safe a table:

urn:uuid:f2504c02-2b2b-11b2-807f-024225278a43    PASSWORD

The user never see the URN.

Storing the password separately is for security.

URNs can be uniquely generated safely with no practical risk of
collision across multiple machines.  SPARQL even has support for it.

    Andy
Post by Lorenz B.
Hello Al,
I do not understand why it is so difficult for you to generate some IRIs
for a users that have an ID. Just use your own namespace and append the
ID. And generating graphs depends on the triple store that you use. In
Jena for instance, you could use some methods of the Dataset [1], like
addNamedModel()
[1]
https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html
Kind regards,
Lorenz
Post by Al Shapiro
A. Soroka,
The syntactical form of my user id values, is a variable containing "U0000001" through "U9999999" - the user id is assigned when an end-user first logs on to the system, with their e-mail and password - as long as the account is valid, the end-user retains the same user id  assigned to it when they first logged on (registered)...
I still need to know how to generate the Named Graph based on IRI, etc?
Thank you,
Al
  What is the syntactical form of your user id values?
---
A. Soroka
The University of Virginia Library
Post by Al Shapiro
Daniel,
I want to create a Named Graph "X", based on User-Id, the data for each named Graph is created on the fly, when a Match occurs against the Default Graph (which has already been loaded to the Jena TDB)...
I need to know how to generate the Named Graph based on IRI?
Do I create a NameSpace for the IRI?
Please give me an example...
Thank you,
Al
I suggest that you first translate your data to RDF using an script
(e.g., in Ruby, Python, etc.) to generate a text file in one of the RDF
serialization formats that support named graphs (e.g., nquads). Then,
you can load the data into jena using tdbloader.
Daniel
Post by Al Shapiro
Daniel,
Thank you, this gives me part of the answer...
Can you give me an example(s) of the IRI's that you used to name your Named Graphs and how they are defined as an IRI, in Sparql?.
Please withhold any proprietary info...
Thank you again,
Al
    Hi Al,
Yes, you can use any IRI to name graphs. Thus, in particular you can use
user ids to compose names of graphs.
Daniel
Post by Al Shapiro
Hi Daniel,
I am a Newbie at Jena Sparql and I would like to know how you addressed the value of each Named Graph you created, for example, I have successfully used GRAPH qm:g1 to create a Named Graph "g1", can I substitute a variable for "g1" such as "user _number", to create multiple Named Graphs based on the value of "user _number"?
Thank you for help...
Al
      Hi Chris,
It is possible. In fact, I have loaded 57 million of named graphs and
then queried without problem. You can see the details in
http://users.dcc.uchile.cl/~dhernand/research/ssws-2015-reifying.pdf
Daniel
Post by Chris Jones
Hi All,
How many named graphs can Jena gracefully handle? Hundreds, thousands,
millions? I've found references to Jena handling large triple stores,
with billions of statements; but nothing about how many named graphs.
I'm writing a system where I'm considering giving each remote client
its own named graph where it uploads data. I'm further considering
keeping historical data, so the client would get a new named graph
each time it sent its data. I'm curious if this is a bad idea.
Chris
Loading...