/prog/ - Legacy of the original /prog/

Name: Anonymous 2014-04-08 23:05

1. The original /prog/, scraped as of august 2013, remains available in sqlite form at the following URL, which, importantly, should still hold even if this site shuts down:

http://web.archive.org/web/*/bbs.progrider.org/files/archives/prog-20130813130608.db.xz

Various reader programs exist.

2. We should have a go at making a final update of the scrape

Name: Anonymous 2014-04-08 23:10

rip 4chan ;_;\

Name: Anonymous 2014-04-08 23:24

How should we go about making a final update?
Do you guys think moot would be willing to give us the /prog/ database?

Name: Anonymous 2014-04-08 23:33

>>3
Full of IP addresses? I hope not. And I double he's willing to spend the effort to expunge it.

Name: Anonymous 2014-04-09 0:02

>>1,3
It's harder to scrape dis.4chan.org now. I think there are limiters that will detect large amounts of requests. But maybe I'm wrong. In any case, the save ratio is around 5%. It's still a precious 5%.

Name: Anonymous 2014-04-09 5:32

http://ask.fm/mrvacbob/answer/111468489368
and there's JSON and Atom APIs to get at the data if you want it

having the newer posts would be nice

Name: Anonymous 2014-04-09 7:07

>>1

What reader programs? Pls respond

Name: Anonymous 2014-04-09 7:18

>>6
The JSON api was deemed crap at some point. I think because of a way it blended poster name and poster trip. There are at least two prog scrapers in existence. Look in the early threads on this board and you may find some.

Name: Anonymous 2014-04-09 10:40

>>7,8
Here's what I have in my progrscrape directory (as base64'd gz'd tar):

Error: Comment too long

Well, okay. http://p.pomf.se/3012

Name: Anonymous 2014-04-09 12:20

>>9
What's with the long strings of AAAAs? What are they like in the original .tar.gz? I can't believe +8 years of /prog/ compress to 700 lines of base64.

Name: Anonymous 2014-04-09 12:29

>>9
Gives me invalid input errors if I try to decode it.

Name: FrozenVoid 2014-04-09 12:41

>>10-11
It's compressed using my infinite compression algorithm.

Name: Anonymous 2014-04-09 16:17

>>10
It's not the .db, it's the scraper program (which some requested), together with a launcher script and a manpage. I'm going to blame >>11's problems on that pasting service apparently liking rewriting newline characters - try sedding them out or something.

Name: Anonymous 2014-04-10 0:26

Xarn has the most up to date scraping of all the data via his world4chsearch.

Name: Anonymous 2014-04-11 13:22

>>13
I was able to decode it properly after piping the text file through ``tr -d '\r\n''' - your suspicions were correct.

Legacy of the original /prog/

1 Name: Anonymous 2014-04-08 23:05

2 Name: Anonymous 2014-04-08 23:10

3 Name: Anonymous 2014-04-08 23:24

4 Name: Anonymous 2014-04-08 23:33

5 Name: Anonymous 2014-04-09 0:02

6 Name: Anonymous 2014-04-09 5:32

7 Name: Anonymous 2014-04-09 7:07

8 Name: Anonymous 2014-04-09 7:18

9 Name: Anonymous 2014-04-09 10:40

10 Name: Anonymous 2014-04-09 12:20

11 Name: Anonymous 2014-04-09 12:29

12 Name: FrozenVoid 2014-04-09 12:41

13 Name: Anonymous 2014-04-09 16:17

14 Name: Anonymous 2014-04-10 0:26

15 Name: Anonymous 2014-04-11 13:22