Somewhat Functional!

gPrime version 1.0.9-alpha2 has been released and the demo updated:

http://demo.gprime.info/  Password: demo, username: demo

Now, you can see 100% of all of the Gramps data model, and edit (without verifications) a good portion of it. You can run all of the reports, and export the data in a variety of formats. Here is the demo with one of the CSS themes (selected under Settings):

  • screenshot-from-2017-01-15-20-46-24

Highlights of this version:

  • GUI editor for notes, saves in Gramps format
  • Can view 100% of Gramps rich schema
    • Can view more than 400 data values
    • Primary and secondary objects in 20 forms
    • 1,400 Python files
    • 291,000 lines of Python code
    • 32 HTML template files
  • Reports can be run and downloaded (PDF format for now)
  • Uses rich gender values
  • Interactive search for selecting parents/spouses on Family view (defaults to gender-specific, but can be toggle to search all, using “+”)
  • Powerful search, and selection language

What’s left to do be a complete system, with adding and editing all data?

  • Implement Undo/Redo
  • Better interface for Report interface
  • Add verification before save, to verify unique values (gid), and proper values
  • Provide better GUI for selecting some values (group as, display as, show as, preformatted)
  • Edit Source on Citation (via interactive search)
  • Edit Place on Event (via interactive search)
  • Add/edit/delete types (name type, origin type, gender type, etc.)
  • Better layout of items on forms
  • A gazillion refinements…

If you have feedback or questions on any aspect of the program, please put them here.

Advertisements

Using gPrime with Gramps

To use gPrime with Gramps, there are points that one should know.

gPrime is centered around the idea of a “site.” A site in gPrime terminology is composed of:

  1. a family tree database
  2. a media directory
  3. a set of user directories

The gPrime database is similar, but not identical to the DB-API Gramps database coming out in Gramps 5.0. So, the best mode of operation is to export your data out of Gramps (using the Gramps XML format) and import that into gPrime. By default, gPrime uses a sqlite database, but you can use any DB-API compliant database (including postgresql, and mysql). The database lives in SITE-DIR/database/ and has a few other files there.

When migrating your genealogy information into and out of gPrime from Gramps, you should always use the Gramps XML format. However, you can use GEDCOM if needed, but gPrime and Gramps can lose some information when going to GEDCOM. For the moment, gPrime and Gramps have exactly the same data models, or schema. However, that could diverge in the future. For example, gPrime will probably add a property on Person to indicate whether or not that they are “probably alive.” This is a useful value, and one that Gramps computes on the fly. This is a possibly expensive cost that can be avoided. As gPrime is designed for very large data files, this is important. The “probably alive” status is useful when exposing your data to others and you want to protect information of living people.

The media directory is designed to be the location for all of the media in gPrime. If you import a family tree, you may have used an absolute path, and those media will continue to be found. If you used a relative path, then you may need to do some copying. The media directory is located in SITE-DIR/media. A cache folder will be created there to save generated images. This is managed by the IIIF Image Server.

screenshot-from-2017-01-12-09-02-31

[UPDATE: as of Jan 12, 2017, gPrime will now automatically attempt to copy media into the media directory. To prevent this attempt, use the –import-media=False flag.]

A set of folders is created for each gPrime “user”. This is so that you can manage the people who have access to your data, and so that each of those people can have their own personal settings (such as theme, report options, etc.)

So, gPrime doesn’t have the ability to create a new family tree, or to switch between them. It could allow making/switching between trees, but that creates complications in managing access. However, many of the reasons why one may have created other family trees in Gramps may be seen as unnecessary with gPrime. For example, the goal is to be able to quickly and effectively create subsets of your data without having to resort to completely different databases. This will be exploited in gPrime 2.0 with the addition of a rich user interface.

Of course, you can still have multiple family trees. But each one will be a different site. In the future, gPrime could serve many sites in a single login. For now, each site requires its own url, users/passwords, and media. Sharing media is of course possible, by just sharing those folders with links. Or, you can just access different sites, one at a time.

Each site is served via a URL. If you are running locally, that doesn’t matter at all. But if you are hosting your family tree with gPrime, then you might want/need to customize your URL. For example, you might need for it to be serve at http://www.myfamily.org/smiths. The “/smiths” is set in gPrime using the –prefix flag.

Status update: gPrime 1.0.9-alpha

As of today (Jan 11, 2017) gPrime can display all data from the Gramps’ data structures! This is a major milestone. There are still a few known bugs, but after fixing those, gPrime will be ready for its beta release. The goal of gPrime 1.0 is to be a viewer of the complete data in the Gramps genealogical model. The next version’s goals (version 1.5) will be a complete editor. Here is demo.gprime.info (username: demo, password: demo) editing a note:

screenshot-from-2017-01-11-21-51-03

I think 1.5 will follow closely on the heels of 1.0, because most everything is already in place for editing, adding, and deleting data.

You can help! At this stage, you could run the demo linked to above, and report back on the following:

  1. Did you see any improper data (the site is hosting Gramps’ example/gramps/example.gramps database)
  2. Did you cause any crashes by just viewing data? There are many things that don’t work with editing data. BTW, you can add some things (like Tags) and you can edit some other things (like Notes).
  3. Do you see how the arrangement of items on the screen could be better?
  4. Did anything seem weird? User unfriendly?

If you like, you can comment here, or you can make an “issue” on gPrime’s issue tracker. If you’d like to discuss a topic, there is also the gPrime Google Group.

Status update: gPrime 1.0.8-alpha

As of today (Jan 7, 2017) gPrime 1.0.8-alpha (username: demo, password: demo) can display all of the primary genealogy data, and most of the rest. The only parts not visible are the more complex subcomponents of objects. These subcomponents come in two types: Common subparts, and references to other primary objects. Specifically:

Common subparts:

  • Address (Repository, Person)
  • Attribute (Person, EventRef, MediaRef, Event, Citation, Source, Family, Media)
  • LDS (Person, Family)

References to other primary objects:

  • ChildRef (Family)
  • EventRef (Person, Family)
  • MediaRef (Person, Family)
  • PersonRef (Person)
  • PlaceRef (Place)
  • RepoRef (Source)

Each of these 9 subcomponents needs a form, handler, and HTML template. Once those are completed, then gPrime 1.0 will be nearly done. gPrime 1.0 will be a complete way to share your genealogy data, and run reports. You can simply import your GEDCOM or Gramps data, and run on a server.

The next step is to be able to edit 100% of all of the data. This is the goal for gPrime 1.5. Once this step is complete, then gPrime will be a working desktop or webserver solution for managing your genealogical data.

However, gPrime 1.5 won’t be as easy to use as, say, Gramps. Gramps has some convenient methods to enter a bunch of data at once. For example, you can enter a family, parents, and children all at one time. With gPrime 1.5, you’ll have to enter each item separately. The plan is to build a new User Interface for gPrime 2.0 that will function more like a desktop application, but better.

gPrime is an Image Server

One of the main goals of the gPrime web-based genealogy software is to integrate it with modern tools. Of course, one requirement for any modern genealogy program is to be able to display images. This could be very simple: “show me a picture!” But, it can also be somewhat more sophisticated: “show be a particular region of a picture!” This requirement looked like a good match with the International Image Interoperability Framework Consortium’s Image API. The IIIF (pronounced “Triple-Eye-Eff”)  Image API:

“…specifies a web service that returns an image in response to a standard HTTP or HTTPS request. The URI can specify the region, size, rotation, quality characteristics and format of the requested image. A URI can also be constructed to request basic technical information about the image to support client applications. This API was conceived of to facilitate systematic reuse of image resources in digital image repositories maintained by cultural heritage organizations. It could be adopted by any image repository or service, and can be used to retrieve static images in response to a properly constructed URI.” From http://iiif.io/api/image/2.1/

A perfect match! It was fairly easy to get their Image API reference implementation to integrate with Gramps’ image region selection parameters. So, you can easily import your Gramps XML data into gPrime and immediately see the proper regions using the Image API.

For example, if you visit (username: demo, password: demo):

http://demo.gprime.info/person/AWFKQCJELLUWDY2PD3

You’ll see a small section of an image to the far right:

screenshot-from-2017-01-04-11-43-16

This image has been extracted and scaled to be exactly the right size for this page, so it isn’t wasting any bandwidth in providing an image that is bigger than necessary. You can enter the URL of that section directly using this appended to the media URL prefix:

/75,93,50,55/80,/0/default.jpg

and get exactly that region:

default

But that is only a portion of the image. To see the whole image, append /full/full/0/default.jpg onto the media URL to see the full image:

default-1

The Image API is actually a mini language for describing image processing manipulations to a picture. For example, you can alter the quality, rotate, scale, select a region, and more. Here is the syntax for the language from http://iiif.io/api/image/2.1/#uri-syntax:

{region}/{size}/{rotation}/{quality}.{format}

You can also get details about the image and the Image API using “/info.json” on the URL:

http://demo.gprime.info/imageserver/238CGQ939HG18SS5MG/info.json

That returns this JSON structure:

{
 "@context": "http://iiif.io/api/image/2/context.json", 
 "@id": "localhost:80/imageserver/238CGQ939HG18SS5MG", 
 "attribution": "Provided by Example Organization", 
 "height": 347, 
 "license": "http://license.example.com/license", 
 "logo": "http://example.com/images/logo.jpg", 
 "profile": ["http://iiif.io/api/image/2/level2.json", 
             {"formats": ["gif", "tif", "pdf"], 
              "qualities": ["color", "gray"], 
              "supports": ["regionSquare", 
                           "canonicalLinkHeader", 
                           "profileLinkHeader", 
                           "mirroring", 
                           "rotationArbitrary", 
                           "sizeAboveFull"]
             }], 
 "protocol": "http://iiif.io/api/image", 
 "sizes": [{"height": 86.75, "width": 125.0}, 
           {"height": 173.5, "width": 250.0}, 
           {"height": 347.0, "width": 500.0}], 
 "tiles": [{"scaleFactors": [1, 2, 4], "width": 512}], 
 "width": 500
}

(Some of that data needs to be properly fleshed out.)

If you want to use your images in Gramps but an image is improperly rotated, you’d have to edit the image to make it display correctly. But using the Image API it could be rotated (even just a fraction of a degree) to display properly. gPrime combined with the Image API gives a complete interface to displaying your images just how you would like. One of features I’d like to incorporate is the functionality of Gramps Photo Tagging gramplet:

This could work in two ways:

  1. Find regions of the image that contains faces
  2. Identify those regions as particular people previously identified in your images

Along those same lines, one could implement a OCR process too. (I work in Machine Learning, and Deep Learning as part of my day job, so I am interested in this.)

The Image API is also designed so that your image collection can be manipulated by third-party applications (using their Presentation API). However, for the gPrime media server to work with that, I think software would need to be created to provide “catalogs”. Not hard, but not high on the goal list either.

The IIIF Consortium is an active group, and are constantly developing new ideas. You can find out more about them here: http://iiif.io/ 

I’d like to thank Ben Brumfield that I met via RootsDev for helping me understand the IIIF Image API. Here is a nice writeup Ben did for RootsDev regarding a hackathon we participated in back in 2015:

https://groups.google.com/d/msg/rootsdev/JWPdCz4An-o/Hi9NfiX-AAAJ

gPrime Demonstration

There is now a gPrime demo available here:

http://demo.gprime.info/

Username: demo
Password: demo

This is an alpha-alpha-alpha version, and so there are many things that don’t work yet. However, there are a few things that do work—at least a little bit.

First, a few points to make you familiar with the demo:

1. All views are shown a page at a time. The page views for each primary object (Person, Family, etc) shows what page you are on, how many pages total, how many items are showing, how many are matching, and the total. For example, if you click here (and log in):

http://demo.gprime.info/person?search=Alvarado

then you will see “Showing 25/26 of 2157”. That means that there are 26 Alavarado’s, 25 showing on this page, out of 2157 total person records. Click on a row to see the details of that row. Click on “>” to go to next page, “<” previous page, “>>” to last page, and “<<” to first page.

screenshot-from-2017-01-02-23-00-13

2. Much of the data in the “example.gramps” data file can now be seen… few items can be edited. First goal is to make everything visible.

3. If there is an error, the system just says “There has been an error. Please check the input and try again.” We’ll have fewer errors, and better messages soon.

4. One of the most interesting working proof-of-concepts is the search mechanism. You can read about it a little here:

https://github.com/GenealogyCollective/gprime/blob/master/docs/Search.md#search

It works in two ways:

a. just type what you want to match, and it will search the default fields for that view in a loose fashion: case insensitive, partial matches.

b. you can search specifically: say exactly what field, and what exactly.

If you enter “Alv“, it will look for that value in a few fields (surname, given, etc). If (in person) you type “surname=Alv%” or “given=Alv%” then it will search for those fields starting with Ala (the percent character is a wildcard, matching anything).

You can search for items that are a bit deep in the object’s structure. To do this you may need to consult the gPrime Schema:

http://htmlpreview.github.io/?https://github.com/GenealogyCollective/gprime/blob/master/docs/Schema.html

For example, to find all of the events (on the events view) from 1987:

date.dateval.2 = 1987

You can also search on joined objects: for example, you can find all the people (on the person view) who have a Death event:

event_ref_list.ref.type.string = Death

This might take a little longer to report the search results, but many searches are possible.

5. Finally, you can combine search with a selection. On the person view, if you say:

select given, surname

select gid, event_ref_list.ref.type.string

You’ll see just the columns you want. You can combine that with a WHERE clause:

select gid, event_ref_list.ref.type.string where event_ref_list.ref.type.string = Death

This selection language is not as easy as it could be yet. But, it should be fairly complete. That is, you should be able to search for any matching criteria that you can think of. (Searching for non-matching values needs some work.)

This is not SQL. This is a special language developed for gPrime that is used throughout the gPrime code to make it easy to program, but fast. gPrime will suport Gramps-style filters too, but for many searches this method may be more efficient.

6. Reports (listed under Actions) can be run, but you can’t see the output yet.

7. The demo password can’t be changed. But you can change the look of the gPrime web interface and the language, under Settings.

screenshot-from-2017-01-02-23-13-03

8. Multiple users can be working simultaneously.

Please feel free to give it a try and report back what you think. Thanks!

Introducing gPrime

gPrime is a new browser-based genealogy program descended from Gramps, and this is a new blog to describe and discuss it. gPrime is designed to become a modern, easy-to-use genealogy program for the 21st century. Specifically it is designed:

  • for collaboration – multi-user ready
  • powerful, easy to use search – select exactly what you want to see with a Google-like text search
  • with security in mind, to keep your information safe and private
  • for very large family trees
  • for future generations
  • for long term and modern technologies – robust database and storage formats
  • for the modern genealogist, including flexible gender categories
  • browser-based interface – works locally, on a server, and on mobile platforms such as Android, iPhones, iPads, etc.

gPrime uses the JSON format for storing all data. This makes your data safer than other programs (Gramps uses “pickled” data which is a security issue, and may not be available to decode for future generations as it has changed over time). gPrime uses modern SQL databases (sqlite, MySQL, and PostgreSQL), although it does not use relational structures.

gPrime is designed so that you can select a subset of any genealogical data based on any criteria. For example, if you want to search common fields, you just enter the information, selected by commas. For example, entering this in the search field:

Smith, Amber

would search common fields for “Smith” and “Amber” and show those that have both. You can also specific which fields to search, like this:

surname=Smith, given=Amber

Finally, you can search for any specific bit of information, if you know how that is specified in the data. For example, imagine that you wanted to find all people who have a father named “Edwin”, then you could enter:

parent_family_list.father_handle.primary_name.first_name=Edwin%

That is a little complicated, but the point is that you do not need to wait for a programmer to add a special filter to search for something, nor do you need a complex set of combinations of filters to select items based on simple values. We’ll try to make this easier, too, over time.

The goal is to make gPrime complete by July 2017, but it is already somewhat usable. See the github software repository for instructions. We’ll cover more details about how to install, test, and help.