# aleph 4.0.0 released

**URL:** <https://aleph.discourse.group/t/aleph-4-0-0-released/118>\
**Category:** Announcements\
**Tags:** release\
**Created:** [October 15, 2024, 10:25am UTC](https://aleph.discourse.group/t/aleph-4-0-0-released/118 "2024-10-15T10:25:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/alephdata/original/1X/1d07899dbb3967bb18027043f2990c4bdd206642.png) [@system](https://aleph.discourse.group/u/system)\
**Post date:** [October 15, 2024, 10:25am UTC](https://aleph.discourse.group/t/aleph-4-0-0-released/118/1 "2024-10-15T10:25:02Z")

</div>

[aleph 4.0.0 released](https://github.com/alephdata/aleph/releases/tag/4.0.0)

Hello Aleph community! We’re excited to announce **Aleph 4.0.0** , a release focused on powerful new features, performance improvements, and expanded options for investigation sharing and user metrics. In addition, this release includes a few other small enhancements, bug fixes and dependency upgrades.

[View full release notes on GitHub](https://github.com/alephdata/aleph/releases/tag/4.0.0)

🚀 Bigger Changes 🚀

- RabbitMQ based task queueing backend
  - Configurable AlephWorker Stages
  - Priority Buckets for Processing
  - System Status Page Enhancements

- Updated Prometheus Metrics
- Documentation Restructure and Enhancements
- Improved Error Handling in Elasticsearch Upgrades

As always, we’d love to hear your feedback to keep improving. Feel free to reach out and share your thoughts!

* * *

## What’s Changed

### Features

#### RabbitMQ

4.0.0 introduces a change to the way background tasks are scheduled. Previously Aleph used a Redis-based task queue, which was well designed but showed its limitations with large payloads and a risk of data loss. RabbitMQ queues are persisted to disk, but the flexibility in the way messages are queued, routed and fetched allows for certain optimizations which Aleph benefits from because of the widely varying degree of task loads.

Migration notes from Redis to RabbitMQ

Due to the significant changes in terms of task status persistence, switching between Aleph versions with RabbitMQ and Redis-based task queues requires some manual steps in order to ensure data consistency.

Perform the following steps every time you are either upgrading to a version with the RabbitMQ task queue or rolling back to the Redis-based task queue:

1. Let all pending jobs run to completion (check the status page).
2. Put Aleph into [maintenance mode](https://github.com/alephdata/aleph/blob/e0d954d6f2182bb2c5bda39bfec6808a41165b71/aleph/settings.py#L31).
3. Stop all workers (`worker`, `ingest-file` processes).
4. (optional) Save the current state of redis in case you want to roll back using the [BGSAVE](https://redis.io/docs/latest/commands/bgsave/) command.
5. Clear Redis (by issuing [FLUSHDB](https://redis.io/docs/latest/commands/flushdb/) from `redis-cli` from the `redis` container). If you get the error message “Unknown command FLUSHDB” then this command is disabled and you can resort to this shell invocation: `echo 'KEYS *' | redis-cli | grep -v '^aleph:' | sed 's/^/DEL /' | redis-cli`.
6. (optional, if previous versions had conflicting RabbitMQ queue settings) Delete existing queues using `rabbitmqctl delete queue {ingest,pruneentity,updateentity,exportxref,analyze,flushmapping,reingest,exportsearch,index,xref,reindex,loadmapping}`. NOTE: queues are named after the stages found in ALEPH\_WORKER\_STAGES.
7. Perform the upgrade or rollback to the desired version of Aleph.
8. Ensure that all expected processes have started correctly.

Related changes:

- Dynamically set AlephWorker stages through env vars by [@catileptic](https://github.com/catileptic) in [#3748](https://github.com/alephdata/aleph/pull/3748)
- Completely remove network diagram embeds feature by [@tillprochaska](https://github.com/tillprochaska) in [#3751](https://github.com/alephdata/aleph/pull/3751)
- Feature: Priority buckets by [@stchris](https://github.com/stchris) in [#3784](https://github.com/alephdata/aleph/pull/3784)
- Separate index worker from other stages in aleph-worker by [@stchris](https://github.com/stchris) in [#3817](https://github.com/alephdata/aleph/pull/3817)

#### Prometheus metrics

We have extended the Prometheus metrics exposed by Aleph to provide more information about active users and the data in your Aleph instance. For example, you can now query for the number of active users within the past 30 days or the number of investigations related to a particular language. For details about the available metrics please refer to the [metrics reference in the technical documentation](https://docs.aleph.occrp.org/developers/reference/prometheus-metrics/).

- New and updated Prometheus metrics by [@tillprochaska](https://github.com/tillprochaska) in [#3844](https://github.com/alephdata/aleph/pull/3844)
- Update Prometheus metrics reference by [@tillprochaska](https://github.com/tillprochaska) in [#3845](https://github.com/alephdata/aleph/pull/3845)
- Fix active users metric by [@tillprochaska](https://github.com/tillprochaska) in [#3852](https://github.com/alephdata/aleph/pull/3852)
- Fix edge cases in custom metrics by [@tillprochaska](https://github.com/tillprochaska) in [#3861](https://github.com/alephdata/aleph/pull/3861)

#### Sharing investigations

Due to the sensitive nature of dataset access we have made some changes to the way datasets are shared, no longer allowing email addresses to autocomplete. This means one needs to know the exact email address of another user if they want to share an investigation.

- Feature: Allow sharing of investigations by [@tillprochaska](https://github.com/tillprochaska) in [#3865](https://github.com/alephdata/aleph/pull/3865)
- Remove sharing options from create investigation screen by [@stchris](https://github.com/stchris) in [#3862](https://github.com/alephdata/aleph/pull/3862)
- Multiple small UX enhancements related to investigation sharing/user suggestion component by [@tillprochaska](https://github.com/tillprochaska) in [#3868](https://github.com/alephdata/aleph/pull/3868)

#### Other new features

- Display start and last updated timestamp on system status page by [@tillprochaska](https://github.com/tillprochaska) in [#3788](https://github.com/alephdata/aleph/pull/3788)
- Display an error message for blocked users by [@tillprochaska](https://github.com/tillprochaska) in [#3560](https://github.com/alephdata/aleph/pull/3560)
- aleph CLI command to downgrade the postgres DB by [@stchris](https://github.com/stchris) in [#3858](https://github.com/alephdata/aleph/pull/3858)

### Bug fixes and other changes

- Use default language when Accept-Language header is ‘\*’ by [@stchris](https://github.com/stchris) in [#3724](https://github.com/alephdata/aleph/pull/3724)
- Exit op\_index early in Aleph Worker by [@catileptic](https://github.com/catileptic) in [#3781](https://github.com/alephdata/aleph/pull/3781)
- Automatically post releases to Discourse by [@stchris](https://github.com/stchris) in [#3728](https://github.com/alephdata/aleph/pull/3728)
- Fix phone numbers used in tests by [@tillprochaska](https://github.com/tillprochaska) in [#3847](https://github.com/alephdata/aleph/pull/3847)
- Remove bookmarks migration by [@tillprochaska](https://github.com/tillprochaska) in [#3752](https://github.com/alephdata/aleph/pull/3752)
- Fix docker compose command by [@tillprochaska](https://github.com/tillprochaska) in [#3843](https://github.com/alephdata/aleph/pull/3843)
- Improve date formats on status page by [@tillprochaska](https://github.com/tillprochaska) in [#3841](https://github.com/alephdata/aleph/pull/3841)
- Aleph upgrade will throw an exception if any ES call returns a status code \< 399 by [@catileptic](https://github.com/catileptic)in [#3859](https://github.com/alephdata/aleph/pull/3859)
- Fix ES index upgrades when using index aliases by [@tillprochaska](https://github.com/tillprochaska) in [#3863](https://github.com/alephdata/aleph/pull/3863)

### 

### Documentation updates

- Docs: Restructure tech docs by [@tillprochaska](https://github.com/tillprochaska)in [#3569](https://github.com/alephdata/aleph/pull/3569)
- Misc documentation enhancements by [@tillprochaska](https://github.com/tillprochaska) in [#3819](https://github.com/alephdata/aleph/pull/3819)
- Docs: added a small amount of text for people looking to get started but are not aware of gitflow by [@Rosencrantz](https://github.com/Rosencrantz) in [#3707](https://github.com/alephdata/aleph/pull/3707)
- Docs: link to download raw docker compose files instead of HTML by [@vsessink](https://github.com/vsessink) in [#3778](https://github.com/alephdata/aleph/pull/3778)
- Docs: Improved order of commands for first time setup by [@vsessink](https://github.com/vsessink) in [#3779](https://github.com/alephdata/aleph/pull/3779)
- Docs: Document how to set up MinIO in a development environment by [@tillprochaska](https://github.com/tillprochaska) in [#3857](https://github.com/alephdata/aleph/pull/3857)
- Docs: Document how to download files using alephclient by [@tillprochaska](https://github.com/tillprochaska) in [#3848](https://github.com/alephdata/aleph/pull/3848)
- Update form links in docs by [@tillprochaska](https://github.com/tillprochaska) in [#3850](https://github.com/alephdata/aleph/pull/3850)
- Update form links on about page etc. by [@tillprochaska](https://github.com/tillprochaska) in [#3851](https://github.com/alephdata/aleph/pull/3851)

### Dependency updates

- Bugfix/downgrade authlib in 3.16.0 by [@stchris](https://github.com/stchris) in [#3574](https://github.com/alephdata/aleph/pull/3574)
- Bump gunicorn[eventlet] from 21.2.0 to 22.0.0 by [@dependabot](https://github.com/dependabot) in [#3689](https://github.com/alephdata/aleph/pull/3689)
- Bump flask and authlib (as required to run flask 3+) by [@stchris](https://github.com/stchris) in [#3732](https://github.com/alephdata/aleph/pull/3732)
- Bump react-pdf from 5.7.2 to 7.7.3 in /ui by [@dependabot](https://github.com/dependabot) in [#3726](https://github.com/alephdata/aleph/pull/3726)

**Full Changelog** : [3.17.0…4.0.0](https://github.com/alephdata/aleph/compare/3.17.0...4.0.0)

---

<div class="post-metadata">

**Author:** ![ezana.ceman](https://yyz2.discourse-cdn.com/free1/user_avatar/aleph.discourse.group/ezana.ceman/32/26_2.png) [@ezana.ceman](https://aleph.discourse.group/u/ezana.ceman)\
**Post date:** [October 15, 2024, 11:01am UTC](https://aleph.discourse.group/t/aleph-4-0-0-released/118/2 "2024-10-15T11:01:03Z")

</div>

Excited to see these new features in action – great work, team! 🚀 🙌

---

<div class="post-metadata">

**Author:** ![alex](https://yyz2.discourse-cdn.com/free1/user_avatar/aleph.discourse.group/alex/32/8_2.png) [@alex](https://aleph.discourse.group/u/alex)\
**Post date:** [October 15, 2024, 11:06am UTC](https://aleph.discourse.group/t/aleph-4-0-0-released/118/3 "2024-10-15T11:06:29Z")

</div>

The transition to RabbitMQ has resulted in a more stable processing of data in Aleph and more insight into what, exactly, Aleph is doing - as revealed by the Status Page.