Dances with Cleanfeed;
Aug. 28th, 2007 10:28 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
serindeor, INN Isn't Done With You Yet, Dearie.
cleanfeed is a perl hook for INN that does its best to ascertain if an article coming in is spam, and if so, then reject it. Life is better with it than without it, but it is not under active development any more, it's hard to find information & discussion about it, and the world has shifted past some of its assumptions.
The problem is with the "phl" filter: this takes a hash of an article's NNTP-Posting-Host and the number of lines in it, and if too many articles come in within a certain space of time with the same hash, it starts rejecting them as likely spam. This is a reasonable approach against things like the legendary Green Card Spam. However, when you have widely-carried hierarchies that are gatewayed through a single server, where people tend to quote entire articles and add one line, your false positive rate is going to go through the roof. microsoft.public.*, I'm looking at you.
Some weeks ago, users who are forced to read those groups complained. When I got a chance, I did a sampling of articles caught by the "phl" test, and I did not find a single one that was actually spam. (Somewhat unsurprising; large-scale Usenet spam is mostly a thing of the past. The big floods are usually by Usenet performance artists, and they do things a bit differently.) So I announced I was turning that filter off, and I did so--it's a simple change in cleanfeed.conf. OR SO I THOUGHT.
The test was still turned on, I saw in the overnight reports. After suitable gibbering, I looked at the actual filter_innd.pl hook. Oh. It wants the local conf file to be in /news/bin/filter, not /news/etc/filter where it actually was. Fine. I popped in a symlink, summoned ctlinnd reload, called it a day.
The test was STILL TURNED ON, I saw in the overnight reports. Upon which I ran up lots of blind avenues, including making sure that we were in fact using the perl invocation, not a python one, &c. Having confirmed this, I looked again at filter_innd.pl and read for comprehension this time; whereupon I learn that it doesn't want cleanfeed.conf--the comments instruct you to tell it where the location of cleanfeed.local is.
But the man page--let me be precise: the man page I had to find in the build directories on juggler, because NO ONE EVER INSTALLED IT ON THE NEWS SERVER--says cleanfeed.conf.
Eventually I realize that said uninstalled man page, though the only one we have in any form, was from an earlier version of cleanfeed, so I copied cleanfeed.conf to cleanfeed.local, reloaded, and am going to call it a day. Until tomorrow when I will find some new, exciting wrinkle.
And now, to beat my head against Mailman.
cleanfeed is a perl hook for INN that does its best to ascertain if an article coming in is spam, and if so, then reject it. Life is better with it than without it, but it is not under active development any more, it's hard to find information & discussion about it, and the world has shifted past some of its assumptions.
The problem is with the "phl" filter: this takes a hash of an article's NNTP-Posting-Host and the number of lines in it, and if too many articles come in within a certain space of time with the same hash, it starts rejecting them as likely spam. This is a reasonable approach against things like the legendary Green Card Spam. However, when you have widely-carried hierarchies that are gatewayed through a single server, where people tend to quote entire articles and add one line, your false positive rate is going to go through the roof. microsoft.public.*, I'm looking at you.
Some weeks ago, users who are forced to read those groups complained. When I got a chance, I did a sampling of articles caught by the "phl" test, and I did not find a single one that was actually spam. (Somewhat unsurprising; large-scale Usenet spam is mostly a thing of the past. The big floods are usually by Usenet performance artists, and they do things a bit differently.) So I announced I was turning that filter off, and I did so--it's a simple change in cleanfeed.conf. OR SO I THOUGHT.
The test was still turned on, I saw in the overnight reports. After suitable gibbering, I looked at the actual filter_innd.pl hook. Oh. It wants the local conf file to be in /news/bin/filter, not /news/etc/filter where it actually was. Fine. I popped in a symlink, summoned ctlinnd reload, called it a day.
The test was STILL TURNED ON, I saw in the overnight reports. Upon which I ran up lots of blind avenues, including making sure that we were in fact using the perl invocation, not a python one, &c. Having confirmed this, I looked again at filter_innd.pl and read for comprehension this time; whereupon I learn that it doesn't want cleanfeed.conf--the comments instruct you to tell it where the location of cleanfeed.local is.
But the man page--let me be precise: the man page I had to find in the build directories on juggler, because NO ONE EVER INSTALLED IT ON THE NEWS SERVER--says cleanfeed.conf.
Eventually I realize that said uninstalled man page, though the only one we have in any form, was from an earlier version of cleanfeed, so I copied cleanfeed.conf to cleanfeed.local, reloaded, and am going to call it a day. Until tomorrow when I will find some new, exciting wrinkle.
And now, to beat my head against Mailman.
