Comments you submit will be routed for moderation. If you have an account, please log in first.
Modify

Ticket #3875 (assigned defect)

Opened 8 years ago

Last modified 7 years ago

Sometimes tickets are not moderated?

Reported by: rasky@… Owned by: jdsiiro
Priority: normal Milestone:
Component: TracTicketModerator Version: TracTicketModerator 0.5.1
Keywords: Cc:

Description

Hello,

I have installed TracTicketModerator 0.5.1 on my trac ( http://www.pyinstaller.org) and did my best so that all submissions are moderated, modulo a few trusted users which I manually flag with MODERATOR_UNMODERATED.

The problem I have is that, even if most spam submissions are correctly moderated, there are a few spam messages which are not. I can't see any pattern on this, and I believe it is a bug of the plugin.

In trac.ini, I put the following in the components section:

ticketmoderator.api.ticketmoderator = enabled
ticketmoderator.web_ui.ticketmoderatormodule = enabled

and then I created a moderator section with the following contents:

[ticketmoderator]
default_moderator = giovannibajo

My trac was just bombed with 3 identical spam, 2 of which have been moderated, while one has passed through without moderation. These are the spam as seen in the "monitoring" section of the admin panel:

 	Path	Author	IP Address	Karma	Date/time
	/ticket/1	 anonymous	93.85.69.151	0	10/26/10 12:58:12
anonymous Излагаются фундаментальные  проблемы ...
	/ticket/155	 anonymous	93.85.69.151	0	10/26/10 12:55:10
anonymous Излагаются фундаментальные  проблемы ...
	/ticket/181	 anonymous	93.85.69.151	0	10/26/10 12:55:01
anonymous Излагаются фундаментальные  проблемы ...

As I said, two of them have been moderator, while one has gone through, as you can see here:
 http://www.pyinstaller.org/ticket/181#comment:2

Can you please let me know how should I debug this problem? It happens about once a week.

Thanks!

Attachments

Change History

comment:1 Changed 8 years ago by jdsiiro

  • Cc jdsiiro@… removed
  • Owner changed from wehart to jdsiiro
  • Status changed from new to assigned
  • Component changed from EXACT to TracTicketModerator

comment:2 Changed 8 years ago by jdsiiro

I have an idea for what may be the root cause... The TicketModerator works by hijacking validate_ticket callbacks. In an attempt to prevent duplicating the warnings sent to the user, the TicketModerator silently "passes" all requests that come in with a warning attached (because, in practice, those warnings get attached as part of the ticket validation process and should prevent the ticket/comment from being accepted). Similarly, the TicketModerator calls the other validate_ticket callbacks and silently returns if any of them return a validation problem.

I think one of two things is occurring:

  • A warning is getting attached to the request outside of trac.ticket.web_ui.TicketModule._validate()
  • Another validator is inconsistent on throwing a validation failure (i.e. it fails validation when the TicketModerator calls it, but not later when the TicketModule? calls it).

comment:3 Changed 8 years ago by jdsiiro

Referenced in changeset [2477]:

Making the TicketModerator more vocal about tickets and attachments that fail validation. This will hopefully address issues where tickets sneak past moderation (see #3875)

comment:4 Changed 8 years ago by jdsiiro

  • Version changed from TracTicketModerator 0.4 to TracTicketModerator 0.5.1

I /think/ that r2477 should fix the problem. Do you mind checking out the trunk and trying it out? Assuming you have setuptools installed, you should be able to build the egg with:

python ./setup.py bdist_egg

comment:5 Changed 8 years ago by rasky@…

Hi, thanks for the quick fix. I have installed the new version from SVN trunk (I ran easy_install on the egg file, and it updated easy_install.pth to point to the new version, so I think the new version is live now).

I'll let you know in the following days if spam messages are still passing through.

comment:6 Changed 8 years ago by rasky@…

Alas, it still does not work. This morning I got 3 spams that passed through spam filters, but only one was held for moderation.

Looking at the logs, I don't see anything glaring. There is no weird traceback or error being generated at the moment the spam has arrived.

If you want to send me a patch to instrument/log more to find out what it is going on, please go ahead. I'm happy also to apply patches against trac itself if it can happen tracking this bug down.

comment:7 follow-up: ↓ 8 Changed 8 years ago by jdsiiro

OK... My next guesses (and I am down to guesses now) are:

  • did the new TicketModerator egg get loaded (e.g., at least if you run mod_python, you need to restart Apache before it will pick up the new module)?
  • there is another plugin or extension that is not properly calling the validation extension point. For example, I know that the email2trac script bypasses ticket validation, and thus will bypass the TicketModerator completely. Would you mind posting the "System Information" section from your "About Trac" page, plus the list of names/versions of your installed plugins? If you are uncomfortable posting that information to this ticket, feel free to email it directly to me.

Unfortunately, I won't be able to look at it at all this week, but I will try and take a closer look next week.

comment:8 in reply to: ↑ 7 Changed 8 years ago by rasky@…

Replying to jdsiiro:

OK... My next guesses (and I am down to guesses now) are:

  • did the new TicketModerator egg get loaded (e.g., at least if you run mod_python, you need to restart Apache before it will pick up the new module)?

I *think* so because we run trac as FastCGI, and I restarted httpd. It would be useful if there was an admin page showing plugin versions, but I guess it's not your fault :)

  • there is another plugin or extension that is not properly calling the validation extension point. For example, I know that the email2trac script bypasses ticket validation, and thus will bypass the TicketModerator completely. Would you mind posting the "System Information" section from your "About Trac" page, plus the list of names/versions of your installed plugins? If you are uncomfortable posting that information to this ticket, feel free to email it directly to me:
Trac:	0.11.7
Python:	2.6.2 (r262:71600, Jun 4 2010, 18:28:04) [GCC 4.4.3 20100127 (Red Hat 4.4.3-4)]
setuptools:	0.6c9
SQLite:	3.6.20
pysqlite:	2.3.5
Genshi:	0.5.1
Pygments:	1.3.1
Subversion:	1.6.13 (r1002816)
jQuery:	1.2.6

I *think* I'm running the following plugins:

TracTicketDelete-2.0-py2.6.egg
TracAccountManager-0.2.1dev_r5836-py2.6.egg
TracSpamFilter-0.2.1dev_r8330-py2.6.egg-info

The SpamFilter has been patched with support for HTTPBL:
 http://trac.edgewall.org/ticket/9133
so before it got committed and went stable.

If you could tell me how a "validation extension point" looks like, I can have a look. I know nothing of Trac plugins API, so I don't know what to search for.

Thanks!

comment:9 Changed 8 years ago by rasky@…

Hi, I'm really flooded by spam... do you have some time to help me debug this problem? I really just need some starting points for debugging. Thanks!

comment:10 Changed 8 years ago by rasky@…

Hi, are you still going to try and help me debug this issue? As explained above, if you could just elaborate on the meaning of "validation extension point" in the context of trac source code, I can analyse this problem myself. Thanks!

comment:11 Changed 8 years ago by jdsiiro

I have looked through the TicketModerator source and have not come up with any ideas on how /some/ tickets can escape moderation. I also run nearly a dozen Trac sites with the plugin enabled, and have yet to see a case where a ticket was able to bypass moderation. So, needless to say, I am stumped.

Here's how the plugin works:

Trac implements an ITicketManipulator extension point. One method available through the extension point is the validate_ticket() method. Trac calls this method before inserting or updating a ticket, allowing plugins the opportunity to tell Trac to reject the ticket. The TicketModerator works by using the call to validate_ticket() to hijack the process and divert the incoming ticket into the moderation queue. The catch is that if validate_ticket() isn't called, then the TicketModerator can't intercept the ticket. For example, this is a known issue with the email2trac script (it calls ticket.insert() directly without ever calling validate_ticket())

So here are my current thoughts on what /might/ be causing the problem:

  • you might be running a plugin or external script that calls ticket.insert() itself without first calling validate_ticket().
  • it might be an issue with thread safety within Trac or the TicketModerator plugin.
    • I tend to run / test against servers using mod_python -- so it could have something to do with how FastCGI works.
    • It might be correlated periods of high load on your server.

comment:12 Changed 7 years ago by jdsiiro

Referenced in changeset [2688]:

Bugfix: change the system for validator recursion detection. This *may* be the cause of some tickets occasionally slipping past the TicketModerator without going through moderation (see #3875).

Adding additional logging information to help with debugging if we ever see tickets that escape moderation again.

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as assigned
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.