antville/antville
antville/antville
1
1
Fork 0
Code Issues 48 Pull requests 17 Projects Releases 9 Wiki Activity Actions 2

Enforce robots.txt rules #502

Merged
tobi merged 1 commit from enforce-robots-txt-rules into main 2025-10-02 19:51:06 +00:00
Conversation 0 Commits 1 Files changed 4 +21 -5
tobi commented 2025-05-27 18:24:33 +00:00
Owner
Copy link
  • Parse the rules of the robots.txt file
  • Check the user agent for a matching rule
  • Respond with status code 401 if the user agent matches
  • Provide a site configuration setting to en/disable the feature
* Parse the rules of the robots.txt file * Check the user agent for a matching rule * Respond with status code 401 if the user agent matches * Provide a site configuration setting to en/disable the feature
tobi added 8 commits 2025-05-27 18:24:33 +00:00
tobi force-pushed enforce-robots-txt-rules from da0726f2d6 to c6d86368c5 2025-05-27 18:24:55 +00:00 Compare
tobi merged commit c6d86368c5 into main 2025-05-27 18:25:25 +00:00
tobi deleted branch enforce-robots-txt-rules 2025-05-27 18:25:25 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
antville.org

   
bug

   
compatibility

   
dependency

Pull request that updates a dependency file

  
duplicate

   
enhancement

   
help wanted

   
invalid

   
java

Pull requests that update Java code

  
javascript

Pull requests that update Javascript code

  
major

   
needs feedback

   
needs work

   
no-issue-activity

   
runtime

   
security

   
urgent

   
usability

   
wontfix
No labels
antville.org
bug
compatibility
dependency
duplicate
enhancement
help wanted
invalid
java
javascript
major
needs feedback
needs work
no-issue-activity
runtime
security
urgent
usability
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: antville/antville#502
No description provided.
Delete branch "enforce-robots-txt-rules"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?