Related to T1901 I also want to make a suggestion how Matrix looks for me.
I've created a general approach which is based on modules.
- We have the frontend which is used by some stakeholders. Those are communicating with the web frontend (in the end with the web backend) or with the web backend REST-API.
- The backend is the gate to the database and processes any logic that's based on direct interaction with Matrix. This means that the user will never communicate directly with pyfunceble.
- The database itself is relational in this approach as I had no better idea. For DB-concept of the versiong system please have a look into T2686#58532. In the overview there's just one database which doesn't mean that we cannot have a separate DBMS for account management or a Redis DB for caching.
- The asset storage is a central place where generated lists stored that are ready for distributing. Assets are then retrieved by the backend on demand. or by other Matrix-related services like the DNS-servers
- As I learned, Pyfunceble is only for checking the availability of domains. This causes a need for another aplication which cares about provisioning the local services, generating assets like RPZ-rules etc. This work could also be done by the web backend but to keep it modular I decided to draw a worker. This worker has an API which the web backend is communicating to. As long as we have only one worker, he can also to the iterating jobs.
- Pyfunceble does the checks for active domains It reports back wheth the collected results about which domains are dead and which cname-mappings (exemptions) are live. This data can be processed in later business products. Inactive domains can be committed over the web backend into the database and then distributed to other projects via web backend.
- The DNS auth server gets the RPZ-rules by the worker. Consumers can then download the RPZ-rules via DNS after they saw that the serial number of the zone changed.
- The DNS-recursor (not on the picture) isn't what I cared about long time but with T1900#58535 in mind I strongly recommend to let only the backend talk with the DNS-server to set ACLs.
The system can be monitored by Icinga and logs can be collected and aggregated with Graylog on Elasticsearch.
For the web backend I recommend SpringBoot (Java framework for web applications) as it has a running instance which results in faster processing and lower query time on the API-side. The web backend can be split into retrieving data from asset storage, backend api and frontend api. This makes maintenance and downtime-handling easier. For the worker I'm not sure if we shall rely on SpringBoot or Python. I've no experience with Python. A plus to Springboot is that it delivers a built in scheduler which would be nice for periodically iterating jobs.