Hi everyone, Klil from OCCRP here with a short update on our progress on this major version release of FtM.
A huge thank you to @pudo for putting together this PR. If you’re interested, I recommended checking out those changes and the previous discourse thread on the subject.
The main feature I want to highlight here is the inclusion of statements-logic in FTM. Previously, Entities were objects where the main unit of analysis was the property value. This means that properties was an object/dict that mapped property names to an array of values. You could query entities on those values, or merge entities by combining these arrays.
In statements, the logic is more robust. Properties is an array of objects, where each contains a property name, value, type, first_seen, last_seen, and other auxiliary information about that expression, (e.g. a Statement). Besides being more expressive, you can do a lot more logically. Statements, as a unit of analysis, can be rows in a database – you can query in granular detail, You can have detailed update logic, etc.
Statements are not enforced, FTM will continue to support Entity fragments, as before. At OCCRP, we continue to use both, in different contexts. But this feature is novel enough to warrant its own thread here.
Right now, @tillprochaska and I are working with Pudo on this PR to make sure that this logic is as generalizable as possible – FTM serves many applications! I don’t know how your databases are organized. FTM should be flexible enough to accommodate all reasonable implementations while still being performant.
If you’re interested in the technical details, please do follow along / contribute on the PR. If you have more general thoughts or questions, feel free to discuss here. We’re always happy to talk to the community.