# Proposal for FollowTheMoney 4.0

**URL:** <https://aleph.discourse.group/t/proposal-for-followthemoney-4-0/158>\
**Category:** Development\
**Tags:** ftm\
**Created:** [January 8, 2025, 5:49pm UTC](https://aleph.discourse.group/t/proposal-for-followthemoney-4-0/158 "2025-01-08T17:49:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pudo](https://yyz2.discourse-cdn.com/free1/user_avatar/aleph.discourse.group/pudo/32/41_2.png) [@pudo](https://aleph.discourse.group/u/pudo)\
**Post date:** [January 8, 2025, 5:49pm UTC](https://aleph.discourse.group/t/proposal-for-followthemoney-4-0/158/1 "2025-01-08T17:49:54Z")

</div>

Hey all, I’d like to pitch a plan for **FollowTheMoney 4.0** that would add several new concepts to the library that we’ve been developing & maturing in `nomenklatura`. The incentive for pushing them up into FtM is to make them easily re-used by future work planned on IDIO’s (@simon) branch of Aleph, and to simplify the class hierarchy we’re using in OS. While the idea is not to break mainstream Aleph, I think some downstream changes would be needed.

- `statements` data model for entity proxies. This essentially makes the unit of property values a `statement` instead a string. The statement captures metadata like the data source, language, timestamps and unparsed value. Besides adding property value metadata, statements also offer an alternative to fragment-based entity aggregation (like in ftm-store), by sorting statements in a given data backend by `entity_id` and then assembling an entity out of them.

- `Dataset` metadata. This could capture all the metadata used in the data catalogs of OS and IDIO. By naming it dataset, it shouldn’t clash with the collections concept in Aleph.

- Remove `Post`. This entity type has been deprecated for over a year, we should be able to remove it now.

There’s a final thing that may or may not make sense to do now-ish, which is to replace the concept of `country` with `jurisdictions` to reflect the fact that an increasing number of the supported values are not countries (examples: Donetsk, Transnistria, Dubai, New York…).

Keen to hear what people think!

---

<div class="post-metadata">

**Author:** ![stchris](https://yyz2.discourse-cdn.com/free1/user_avatar/aleph.discourse.group/stchris/32/6_2.png) [@stchris](https://aleph.discourse.group/u/stchris)\
**Post date:** [January 14, 2025, 11:58am UTC](https://aleph.discourse.group/t/proposal-for-followthemoney-4-0/158/2 "2025-01-14T11:58:10Z")

</div>

Hi @pudo !

First of all thanks for the detailed write-up! From the Aleph side of things we are still behind on ftm versions, so we’ll need some more time to catch up. Everything you wrote above makes sense to me and I have nothing against moving forward with these changes.

---

<div class="post-metadata">

**Author:** ![pudo](https://yyz2.discourse-cdn.com/free1/user_avatar/aleph.discourse.group/pudo/32/41_2.png) [@pudo](https://aleph.discourse.group/u/pudo)\
**Post date:** [April 28, 2025, 1:15pm UTC](https://aleph.discourse.group/t/proposal-for-followthemoney-4-0/158/3 "2025-04-28T13:15:49Z")

</div>

A quick update on this:

- [#1727](https://github.com/alephdata/followthemoney/pull/1727) brings in the dataset and data catalog models from `nomenklatura`, as proposed above.
- [#1728](https://github.com/alephdata/followthemoney/pull/1728) depends on #1727 and brings in the `Statement` class, as well as a subclass of `EntityProxy`, `CompositeEntity`, which is an entity based on statements. Because each `Statement` has a `dataset` field, this is linked to the previous. This also introduces new CLI commands `statements` (turns entities into a statement feed), `format-statements` (converts statements between CSV and JSON form), and `aggregate-statements` (which turns a sorted stream of statements back into entities).
- [#1733](https://github.com/alephdata/followthemoney/pull/1733) consolidates a medium-aggressive set of schema changes for 4.0, removing `Post` and `Assessment` and re-naming some of the naming clashes discussed in [#1732](https://github.com/alephdata/followthemoney/issues/1732). I’ve left some of the gnarly overlaps in the schema for now, because fixing the would necessitate large-scale data migration.

Then there’s two smaller PRs: one removes the dependency on `fingerprints` from `followthemoney`: the functionality of that library is now fully contained in `rigour`. The other PR consolidates all the code related to RDF exports into one module. Part of the idea here is that not every invocation of `ftm` should require loading `openpyxl` and `rdflib`, two fat dependencies.

What I’d still like to do before making this a release:

- Bring in a `Risk` schema, [#1723](https://github.com/alephdata/followthemoney/issues/1723)
- Would love to see more work on formalising `numbers` and `units`
- Merge in @Keden’s `proof` PR.
- Create `property.caption()` as discussed in the other thread.

Soon it may make sense to start doing a `develop` branch for all these changes to be collected, which could eventually become `4.0`.

Keen for any feedback!

---

<div class="post-metadata">

**Author:** ![pudo](https://yyz2.discourse-cdn.com/free1/user_avatar/aleph.discourse.group/pudo/32/41_2.png) [@pudo](https://aleph.discourse.group/u/pudo)\
**Post date:** [May 4, 2025, 1:43pm UTC](https://aleph.discourse.group/t/proposal-for-followthemoney-4-0/158/4 "2025-05-04T13:43:16Z")

</div>

Ok, so there’s a `develop` branch here now: [Develop (FtM 4.0) by pudo · Pull Request #1749 · alephdata/followthemoney · GitHub](https://github.com/alephdata/followthemoney/pull/1749)
