The Archiving of Topics

[JARG]

The parameters are such: Any thread that hasn't (in most forums) seen a new post in 13 years, does not contain a poll, and isn't pinned, will go into archive mode. Those are pretty conservative rules, but since a lot of content was posted during the early days, archiving the first four years clears out about 35% of our total threads. Archiving moves those threads to a different database, lessening the load on the main one. When you have a long-standing community like ours with a lot of posts (approaching 5 million), at some point you have to do some pruning.

 

Yes, fetching records out of a set of 5 million is going to take some time, regardless of indexing. And yes, your parameters are extremely conservative. I have a ticketing tool app that archives cases 30 days after their closure date.

 

Are all the posted comments stored in a single table or does each sub-forum have a dedicated table?

[1-0-0-1-0-0-1]

The parameters are such: Any thread that hasn't (in most forums) seen a new post in 13 years, does not contain a poll, and isn't pinned, will go into archive mode. Those are pretty conservative rules, but since a lot of content was posted during the early days, archiving the first four years clears out about 35% of our total threads. Archiving moves those threads to a different database, lessening the load on the main one. When you have a long-standing community like ours with a lot of posts (approaching 5 million), at some point you have to do some pruning.

 

Yes, fetching records out of a set of 5 million is going to take some time, regardless of indexing. And yes, your parameters are extremely conservative. I have a ticketing tool app that archives cases 30 days after their closure date.

 

Are all the posted comments stored in a single table or does each sub-forum have a dedicated table?

 

Single table. I don't know enough about this stuff to know if that's advantageous or not.

[Babycat]

*does an impression of Indiana Jones tossing spare triple strength catnip, toys, bed, and scratching post out just before the door slides down*

 

Heh... don't want to leave any of that behind..! :lol:

[JARG]

The parameters are such: Any thread that hasn't (in most forums) seen a new post in 13 years, does not contain a poll, and isn't pinned, will go into archive mode. Those are pretty conservative rules, but since a lot of content was posted during the early days, archiving the first four years clears out about 35% of our total threads. Archiving moves those threads to a different database, lessening the load on the main one. When you have a long-standing community like ours with a lot of posts (approaching 5 million), at some point you have to do some pruning.

 

Yes, fetching records out of a set of 5 million is going to take some time, regardless of indexing. And yes, your parameters are extremely conservative. I have a ticketing tool app that archives cases 30 days after their closure date.

 

Are all the posted comments stored in a single table or does each sub-forum have a dedicated table?

 

Single table. I don't know enough about this stuff to know if that's advantageous or not.

 

I think it's probably fine. Having them all in one table certainly makes house-keeping tasks, such as archiving, easier. The most obvious drawback is that the table has a buttload of rows.

[1-0-0-1-0-0-1]

Archived

New World Women

One Little Victory

Tour Meetup Archives

Reverse Polarity

Music Of The Spheres

 

Currently Archiving

Random Samples

 

5675 threads archived so far in five days. RS by itself will archive 10432, and will probably take over a week to complete. .

 

RS is one of our biggest forums and can take a long time to load up, and there might be times where it takes even longer as the older threads are being moved off of the main database.

[1-0-0-1-0-0-1]

Archived

New World Women

One Little Victory

Tour Meetup Archives

Reverse Polarity

Music Of The Spheres

 

Currently Archiving

Random Samples

 

5675 threads archived so far in five days. RS by itself will archive 10432, and will probably take over a week to complete. .

 

RS is one of our biggest forums and can take a long time to load up, and there might be times where it takes even longer as the older threads are being moved off of the main database.

 

The archiving of RS is ongoing, and going pretty well after a routine repair of the database a few nights ago. RS, GD, SOCN and Rush will take a long time to archive. The goal is to archive about 35% of the total threads on TRF, and right now we're at 6.65%. Hopefully we'll see the board run better as this goes on.

[1-0-0-1-0-0-1]

I'm toying with the idea of closing down Grand Designs for a few days so it can be properly archived. What I'm noticing is the larger threads (1,000+ posts) take a really long time to process, and each time it finishes up one of those threads and migrates it off of the main database it bogs the board down (which is what I'm noticing with the large threads being archived in OLV right now). Some of the TOST threads in GD have over 10,000 posts -- it might take half a day to do just one of those threads. Maybe if no one was accessing that forum during that process it might go faster and with less negative impact on the rest of the board?

[JohnRogers]

I'm toying with the idea of closing down Grand Designs for a few days so it can be properly archived. What I'm noticing is the larger threads (1,000+ posts) take a really long time to process, and each time it finishes up one of those threads and migrates it off of the main database it bogs the board down (which is what I'm noticing with the large threads being archived in OLV right now). Some of the TOST threads in GD have over 10,000 posts -- it might take half a day to do just one of those threads. Maybe if no one was accessing that forum during that process it might go faster and with less negative impact on the rest of the board?

Do what you gotta do, Boss. I can chat in TOST thread at The Duran Duran Forum for a few days.

[1-0-0-1-0-0-1]

The archiving of Grand Designs is ongoing, very slowly. I started with the threads with the least amount of posts and I'm working my way up.

 

GD contains the biggest threads on the board, and there's a bunch of them. The closed TOST threads are very big (the biggest has 8512 posts), but the biggest in GD are the closed A-Z word game threads, with five of them eclipsing 10,000 posts. Archiving just one of those threads could take a whole day, maybe longer, and the process of archiving one thread that size would slow the board down for minutes at a time, all day long. The biggest has 18936 posts! Imagine how long that one would take.

 

The thing is, do we need to archive those game threads? We really don't. Therefore, they will be deleted. We really only need the current version of any game thread -- once a closed version has been replaced with a new one, I say out with the old. Two seconds to delete a 10,000+ post word game thread that no one will go back and read as opposed to over a day to archive it? Yes, that makes too much sense.

 

The plan is to archive the closed TOST threads. A few years ago I deleted a bunch of the really old ones, but we'll try and keep the ones that remain. Unless you regular TOSTers feel like we can lose them. If you guys feel like you'll never go back and read those old threads, please let me know.

 

Once I get ready to start archiving the big threads, I think I'll move the most popular GD threads out to RS so people can continue to use them, then temporarily close GD and finish archiving it. I'm thinking cutting off traffic to that forum will smooth out the process. Then I'll re-open it and move those threads back in.

 

And to answer the question some of you were about to frantically type before reading this sentence, no, deleting threads does NOT lower your post count. :fistbump:

[Mr. Not]

The plan is to archive the closed TOST threads. A few years ago I deleted a bunch of the really old ones, but we'll try and keep the ones that remain. Unless you regular TOSTers feel like we can lose them. If you guys feel like you'll never go back and read those old threads, please let me know.

 

 

personally the only TOST threads i'd ever care to ever look back on are the absolute oldest ones which are already deleted, and even then i honestly probably wouldn't bother either.

if it helps the board to the delete more recent ones as well then i say go for it

[JohnRogers]

The plan is to archive the closed TOST threads. A few years ago I deleted a bunch of the really old ones, but we'll try and keep the ones that remain. Unless you regular TOSTers feel like we can lose them. If you guys feel like you'll never go back and read those old threads, please let me know.

 

 

personally the only TOST threads i'd ever care to ever look back on are the absolute oldest ones which are already deleted, and even then i honestly probably wouldn't bother either.

if it helps the board to the delete more recent ones as well then i say go for it

Are there some fun and interesting discussions in them? Interactions with people no longer around? Sure but if a cost/benefit analysis was done you’ll find deleting them makes sense.

[Mr. Not]

The plan is to archive the closed TOST threads. A few years ago I deleted a bunch of the really old ones, but we'll try and keep the ones that remain. Unless you regular TOSTers feel like we can lose them. If you guys feel like you'll never go back and read those old threads, please let me know.

 

 

personally the only TOST threads i'd ever care to ever look back on are the absolute oldest ones which are already deleted, and even then i honestly probably wouldn't bother either.

if it helps the board to the delete more recent ones as well then i say go for it

Are there some fun and interesting discussions in them? Interactions with people no longer around? Sure but if a cost/benefit analysis was done you’ll find deleting them makes sense.

 

yes there's still threads with polarizeme and homesickalien interactions for example and neither of them will be coming back

 

while sifting through them for old time's sake was something i used to do i'd never do it now - literally just too much spam to sift through in order to find anything of value. it makes sense that if we want serious discussions to look back on it should be outside of TOST.

 

another change in TOST i've noticed over time (or maybe i'm just believing this changed because of my relative inactivity) is that we don't really have inside jokes anymore. it's more of just a casual place to chat and let out thoughts now

[HomesickAlien]

The plan is to archive the closed TOST threads. A few years ago I deleted a bunch of the really old ones, but we'll try and keep the ones that remain. Unless you regular TOSTers feel like we can lose them. If you guys feel like you'll never go back and read those old threads, please let me know.

 

 

personally the only TOST threads i'd ever care to ever look back on are the absolute oldest ones which are already deleted, and even then i honestly probably wouldn't bother either.

if it helps the board to the delete more recent ones as well then i say go for it

Are there some fun and interesting discussions in them? Interactions with people no longer around? Sure but if a cost/benefit analysis was done you’ll find deleting them makes sense.

 

yes there's still threads with polarizeme and homesickalien interactions for example and neither of them will be coming back

 

 

 

Such pessimism

[Mr. Not]

The plan is to archive the closed TOST threads. A few years ago I deleted a bunch of the really old ones, but we'll try and keep the ones that remain. Unless you regular TOSTers feel like we can lose them. If you guys feel like you'll never go back and read those old threads, please let me know.

 

 

personally the only TOST threads i'd ever care to ever look back on are the absolute oldest ones which are already deleted, and even then i honestly probably wouldn't bother either.

if it helps the board to the delete more recent ones as well then i say go for it

Are there some fun and interesting discussions in them? Interactions with people no longer around? Sure but if a cost/benefit analysis was done you’ll find deleting them makes sense.

 

yes there's still threads with polarizeme and homesickalien interactions for example and neither of them will be coming back

 

 

 

Such pessimism

 

lol - only took 4 years

[Babycat]

Got to admit, my The Last One Who Posts Wins v2 did well for almost 11 years, just short by one month. Well done, 73 for winning! :clap: :clap: :clap:

[Mr. Not]

 

morning boss, this seems like the place to address this

 

when searching for TRF threads via google i often get hit with a 404 page when trying to access them, and when i use the in-site search function i can only access the first page of results, when i try to go "next" or jump to another page, i'm also hit with a 404.

 

is this King's X thread still accessible? I need to know before starting a new one

http://www.therushforum.com/index.php?/topic/39356-kings-x/

 

it's over 13 years old but it definitely was a semi-active thread through the years

[1-0-0-1-0-0-1]

 

morning boss, this seems like the place to address this

 

when searching for TRF threads via google i often get hit with a 404 page when trying to access them, and when i use the in-site search function i can only access the first page of results, when i try to go "next" or jump to another page, i'm also hit with a 404.

 

is this King's X thread still accessible? I need to know before starting a new one

http://www.therushfo.../39356-kings-x/

 

it's over 13 years old but it definitely was a semi-active thread through the years

 

The Google search result link is something we're going to fix very soon. Google thinks we're https, but we're still http, so when you click their link you get nothing. Hopefully within a week or two we'll have that done. That should have been updated a couple of years ago, but I had no idea the search engine links were doing that. That's on me, I probably should have known that.

 

That Kings X thread link still works.

[Mr. Not]

^ lol: http://www.therushfo...x/#entry3652647

 

swear there was a king's x thread with more pages as well..

 

i'll do some digging but it's looking like i'll have to start anew

Edited March 30, 2022 by Mr. Not

[1-0-0-1-0-0-1]

^ lol: http://www.therushfo...x/#entry3652647

 

swear there was a king's x thread with more pages as well..

 

i'll do some digging but it's looking like i'll have to start anew

 

I'm pretty sure Fridge or WCFIELDS started one in MoTS, but the goddamn search function on this board is useless right now. :(

[Mr. Not]

^ lol: http://www.therushfo...x/#entry3652647

 

swear there was a king's x thread with more pages as well..

 

i'll do some digging but it's looking like i'll have to start anew

 

I'm pretty sure Fridge or WCFIELDS started one in MoTS, but the goddamn search function on this board is useless right now. :(

 

Yeah, that thread / Fridge is what i remember too, Fridge got me into them to begin with and he also had this cool photo with the band at a meet and greet - they all looked goofy and mismatched during that year.

 

i'll wait it out then

[1-0-0-1-0-0-1]

^ lol: http://www.therushfo...x/#entry3652647

 

swear there was a king's x thread with more pages as well..

 

i'll do some digging but it's looking like i'll have to start anew

 

I'm pretty sure Fridge or WCFIELDS started one in MoTS, but the goddamn search function on this board is useless right now. :(

 

I imagine MoTS is one of the forums that gets a lot of searches. I just looked at the threads in there sorted by highest post counts, and on the first page are 17 closed "Listening to" threads. These are the post counts in each:

 

27775 replies

22481 replies

19997 replies

19983 replies

16033 replies

14313 replies

13355 replies

11910 replies

11379 replies

11181 replies

10953 replies

10004 replies

9663 replies

9284 replies

7206 replies

6722 replies

5565 replies

 

Total: 193922

 

Almost 200,000 posts in 17 threads that no one will ever look at again. Here I am concentrating on GD as far as closed game threads and MoTS is a monster.

[HomesickAlien]

nvm Edited March 30, 2022 by HomesickAlien

[HomesickAlien]

nvm

 

That did not post correctly at all. Oh, well. Try again later. :eh: