Multi-server environment with Crowdsec

With this indigestible title, let's try to explain what makes Crowdsec a brown beast… as it will be usual to have the Crowdsec agent on each of our machines in the organization, We will make it possible for them to communicate with each other to prevent attacks.

This is, than if a Windows or Linux computer is attacked, the rest of the organization's machines can find out about the attack and block the attacker's IP address before they can even knock on their door. By default, let's say that Crowdsec works locally, each Crowdsec agent communicates to its own Local API server, what we will do is that they all communicate to the same LAPI server.

Local API Server,

Started, first, on the machine we want to act as an API server for the rest of the agents. So we enable it in your configuration file ‘ /etc/crowdsec/config.yaml', we indicate the IP by which it will listen and the port:

API:
...
  Server:
...
    listen_uri: 192.168.2.10:8080
...

In '/etc/crowdsec/local_api_credentials.yaml’ only the same IP address:

URL: HTTP://192.168.2.10:8080
...

We restart to apply the changes:

sudo systemctl restart crowdsec

Crowdsec Agents,

Now, the rest of the machines with Crowdsec must be registered against the Local API of the server, and we will also disable the LAPI of each Crowdsec machine since it will not be used.

To put it offtopic, if you do not have Crowdsec installed on the target machine to be protected, It's time 😉 You have in previous posts how to protect a Windows, Linux…

Well, Come, goes, we register against the LAPI server from each agent with:

Sudo cscli lapi register -u HTTP://192.168.2.10:8080
    INFO[14-11-2022 10:14:08 PM] Successfully registered to Local API (LAPI)
    INFO[14-11-2022 10:14:08 PM] Local API credentials dumped to '/etc/crowdsec/local_api_credentials.yaml'
    WARN[14-11-2022 10:14:08 PM] Run 'sudo systemctl reload crowdsec' for the new configuration to be effective.

We stop our own API server as we won't use it:

sudo cp /lib/systemd/system/crowdsec.service /etc/systemd/system/crowdsec.service

And we add '-no-api’ in the Crowdsec configuration file '/etc/systemd/system/crowdsec.service’

...
ExecStart=/usr/bin/crowdsec -c /etc/crowdsec/config.yaml -no-api
...

We finally reload the services to use this new configuration:

sudo systemctl daemon-reload sudo systemctl restart crowdsec

Now if we go to the LAPI server, We will be able to see that there are pending connection requests with the command 'sudo cscli machines list':

---------------------------------------------------------------------------------------------------------
 NAME IP ADDRESS LAST UPDATE STATUS VERSION AUTH TYPE LAST HEARTBEAT
---------------------------------------------------------------------------------------------------------
 b91d17c64c4e4a2 192.168.x.xxx 2022-11-14T22:18:00Z ✔️ v1.4.1-debian password 19s 9c70ab3970dd4cc 192.168.x.xxx 2022-11-14T22:14:08Z 🚫 password ⚠️ 4m11s
---------------------------------------------------------------------------------------------------------

We validate the machines by running 'sudo cscli machines validate 9c70ab3970dd4cc’

sudo cscli machines validate 9c70ab3970dd4cc INFO[14-11-2022 10:19:35 PM] machine '9c70ab3970dd4cc' validated successfully

And we check again 'sudo cscli machines list':

---------------------------------------------------------------------------------------------------------
 NAME IP ADDRESS LAST UPDATE STATUS VERSION AUTH TYPE LAST HEARTBEAT
---------------------------------------------------------------------------------------------------------
 B91D17C64C4E4A 192.168.x.xxx 2022-11-14T22:19:00Z ✔️ v1.4.1-debian password 40s 9c70ab3970dd4cc 192.168.x.xxx 2022-11-14T22:19:35Z ✔️ password 5s
---------------------------------------------------------------------------------------------------------

Not forgetting to top up Crowdsec in each agent once we have validated it, with:

sudo systemctl restart crowdsec

Mitigations,

Now, to mitigate, to implement actions that allow us to reduce the risks of vulnerability to certain threats, We'll have to use the bouncers. First, from the API server we will create the tokens that the agent bouncers need. We will do it with every bouncer we need in every agent, example 'sudo cscli bouncers add OS-JITSI-05-Fw’

Api key for 'OS-JITSI-05-fw':
    xxxxxxxxxxxxx Please keep this key since you will not be able to retrieve it!

In each Crowdsec agent we must install the bouncer we need, In this case we will rely on firewall-type bouncers, for each OS, There is a compatible bouncer: Windows firewall, iptables, Unknown, ipset, Pf… If we have iptables:

sudo apt install crowdsec-firewall-bouncer-iptables -y

And we edit the firewall bouncer configuration file ‘ /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml’ indicating the IP address of our LAPI server and the token we have just generated for it:

...
api_url: HTTP://192.168.2.10:8080/
api_key: xxxxxxxxxxxx
...

We restart the firewall bouncer service:

Sudo SystemCTL Restart CrowdSEC-Firewall-Bouncer sudo SystemCTL Status CrowdSEC-Firewall-Bouncer

And that's it! This machine will be ready, you can add entries to the OS firewall by denying access to certain IP addresses communicated to you by the API server.

Podremos desde el servidor LAPI verificar el estado de los bouncers con ‘sudo cscli bouncers list’:

-----------------------------------------------------------------------------------------------------------------------
 NAME                IP ADDRESS     VALID  LAST API PULL         TYPE                       VERSION          AUTH TYPE
-----------------------------------------------------------------------------------------------------------------------
 OS-JITSI-05-fw      192.168.x.xxx  ✔️      2023-02-02T16:40:16Z  crowdsec-firewall-bouncer  v0.0.24-debian  api-key  OS-ELK-03-fw        192.168.x.xxx  ✔️      2023-02-02T16:40:11Z  crowdsec-firewall-bouncer  v0.0.24-debian  api-key  OS-GRA-04-fw        192.168.x.xxx  ✔️      2023-02-02T16:40:11Z  crowdsec-firewall-bouncer  v0.0.24-debian  api-key  OS-NGINX-01-fw      192.168.x.xxx  ✔️      2023-02-02T16:40:10Z  crowdsec-firewall-bouncer  v0.0.24-debian  api-key  OS-NGINX-01-mirror  192.168.x.xxx  ✔️      2023-02-02T16:40:18Z  crowdsec-blocklist-mirror  v0.0.1-9-g86d6  api-key  OS-OTRS-01-fw 192.168.x.xxx ✔️ 2023-02-02T16:40:16Z crowdsec-firewall-bouncer v0.0.24-debian api-key
...
------------------------------------------------------------------------------------------------------------------------

And that's it! With this, the mesh is closed! All of our Crowdsec in the organization will be spoken to using a single API server, so everyone will be equally protected, sharing a single black IP address list, blocking attackers' IPs in their respective FWs…

Hugs! May it go VERY well 😉

Recommended Posts

Author

nheobug@bujarra.com
Autor del blog Bujarra.com Cualquier necesidad que tengas, Do not hesitate to contact me, I will try to help you whenever I can, Sharing is living ;) . Enjoy documents!!!