Actions

Property

Has bot segment

From WikiApiary, monitoring the MediaWiki universe

Has bot segment is a Number and is assigned by Template:Website. The purpose of this property is to group Category:Websites into a small number of groups for bots to separately collect information for, parallelizing the statistics collection. As the number of websites increases above several hundred, User:Bumble Bee will operate better by doing multiple sites concurrently. Rather than introducing a lot of complexity with multithreaded programming in the bot, we instead just group the sites into multiple segments and the scripts can then deal with subsets of the sites concurrently, with multiple processes.

This property using a modulo of the PAGEID by the number of bots running. There are currently 60 bot segments, which is set by changing the value at WikiApiary:Bot segments. The statistics bots then run in parallel via cron, with each run taking a different segment. These may also be offset in cron so they aren't running all at once. One approach using 6 segments would be to have two running in parallel every 5 minutes, repeating the loop every 15 minutes.

See current bot segments for current breakdown of segments.

Related

Pages using the property "Has bot segment"

Showing 25 pages using this property.

View (previous 25 | next 25) (20 | 50 | 100 | 250 | 500)

.
.5kompetition +43  +
.NET Micro Framework Wiki +30  +
.hr domene +9  +
/
/usr/space Wiki +57  +
0
001 Wiki +27  +
007 Fanon Wiki +28  +
007Film. Wiki +29  +
027 +45  +
03.com.ua +2  +
07n +20  +
0x3b Project +3  +
1
1 to 101 +11  +
1,5 Liter Flasche - Wiki +21  +
100 Startup Ideas +53  +
100 Wars +2  +
1000BlankWhiteCardsWiki +29  +
100MonkeysResource +57  +
101 Dalmatian Street Wiki +29  +
101 Dalmatians Wiki +37  +
118Wiki +13  +
1217 +58  +
123 +16  +
123 Flash Chat wiki +10  +
1337 +40  +
13Lights +24  +