Why do so many programs use rational databases instead of loading the data during startup and keeping it in memory? Especially for smaller datasets I would think, that a database adds unnecessary complexity and overhead. Also, a lot of data can be saved using modern RAM and when using an in-memory approach, optimized data structures can be utilised to further improve the performance
I presume you’re referring to relational databases instead of rational.
The responsibility of a RDBMS is to implement a set of performance-focused data structures that help clients reliably get the data that they need in the fastest possible way, without having to reinvent the wheel.
More often than not, this data does not fit in the heap.
Also, in many usecases there is more than a single client.
Hope this helps.