Blacklists are empty after filter sync

[psalecker]

I'm using CrowdSec on pfSense Plus 24.03 and noticed that the crowdsec_blacklists and crowdsec6_blacklists PF tables are often empty, while the crowdsec-firewall-bouncer is working as it should. I narrowed the problem down to the filter_configure_sync function of pfSense.

The easiest way to reproduce it is to call /etc/rc.filter_configure_sync:

# /sbin/pfctl -t crowdsec_blacklists -T show | wc -l
   56019
# /etc/rc.filter_configure_sync
755 addresses deleted.
56019 addresses deleted.
# /sbin/pfctl -t crowdsec_blacklists -T show | wc -l
       0

In practice I saw two use-cases where this happens:

1. When having have two pfSense instances and Configuration Synchronization configured, every time you save a change on the primary instance, it reloads everything on the secondary instance, causing the tables to get cleared on the secondary.
2. When using Time Based Rules, pfSense adds a line to the /etc/crontab, that is clearing the tables every 15 minutes:
   0,15,30,45 * * * * root /etc/rc.filter_configure_sync

Ideally the CrowdSec package would hook into the filter_configure_sync function and either prevents that the entries get removed or adds them back immediately.

[FlurryNight]

Hey.

Having same issue @psalecker , plus rules are not being createad at all 🥲..

Using pfSense Plus 24.11.

However before in 24.03 i tested with manual decision and it worked and blocked the traffic.

But now, it does not block anything. And also the table gets empty.

@mmetc Thanks for your work. If you need to debug further. let me know.

Best regards. And Happy Holidays

[mmetc]

@FlurryNight have you installed this pre-release version? https://github.com/crowdsecurity/pfSense-pkg-crowdsec/releases/tag/v0.1.5-1.6.4pre

can you please run "cscli support dump" and send the resulting file to support@crowdsec.net

Are you also using two pfsense instances?

[FlurryNight]

@FlurryNight have you installed this pre-release version? https://github.com/crowdsecurity/pfSense-pkg-crowdsec/releases/tag/v0.1.5-1.6.4pre

can you please run "cscli support dump" and send the resulting file to support@crowdsec.net

Are you also using two pfsense instances?

Hello, yes. I also have upgraded to see if it would solve. Previously i was using stable.

Okay i will do that soon as i can.

No. Only one.

Thanks

[FlurryNight]

@mmetc

I've sent the email :).

Thanks

[mmetc]

Hi @FlurryNight

I have not been able to replicate your issue -- the filter table is only empty when the bouncer is not running, but that's not what I see from the logs.

You can try the new version at https://github.com/crowdsecurity/pfSense-pkg-crowdsec/releases/tag/v0.1.5-1.6.4pre2

if it's still not working we can switch to email for more details

thanks

[FlurryNight]

Hello,

Thanks for the response.

I've upgraded but no luck.. i also can't exactly point why..

Another thing i noticed from the start, is after enabling the crowdsec package in settings or doing pkg upgrades i had to do
sudo service crowdsec onereload
and
sudo cscli capi status

for it to be detected in the Crowdsec Platform and to function okay. don't know if that is a issue too.

However since i upgraded pfSense to 24.11 i dont know why the rules are not being created, and since the tables get empty it not possible to do the manual rules. the thing is like you said, all seems normal. bouncer is running and etc.

Could it be from some change from netgate in this last version?

We can do a call in Discord now in January if you need to debug live, if you have discord contact me via the email i've sent. and i send you my tag.

Thanks again, and have a happy new year

[nnaghawai]

Hi all, I bumped into this issue just now, not sure if this was fixed already (I've installed the latest), as a quick workaround I added a cron job that restarts the bouncer if the table is empty

* * * * * root [ `/sbin/pfctl -t crowdsec_blacklists -T show | wc -l | awk '{print $1}'` -eq 0 ] && service crowdsec_firewall onerestart

[FlurryNight]

Hi all, I bumped into this issue just now, not sure if this was fixed already (I've installed the latest), as a quick workaround I added a cron job that restarts the bouncer if the table is empty

* * * * * root [ `/sbin/pfctl -t crowdsec_blacklists -T show | wc -l | awk '{print $1}'` -eq 0 ] && service crowdsec_firewall onerestart

Hi, it is not fixed still. tried to arrange a call for debugging. but got no answer...

Much thanks for the workaround. will try it!