דטאבייס של 140 אלף קישורי הורדות (ראפידשר מגה אפלואד וכו')

[intercooler3819]

פרסמתי את זה באנגלית אז אני פשוט יצטט

Hi,
I have made a robot 2 weeks ago which fetchs and index'es rapidshare, megaupload and other free-upload websites download links into database table.
It runs automaticly each 5 hours and collect the links from about 20-30 websites.
In 2 weeks I have this:
100,000 links, indexed in mysql table.
The table takes 20MB of space
The table scheme is like this:

-- --------------------------------------------------------

--
-- Table structure for table `search`
--

CREATE TABLE IF NOT EXISTS `search` (
  `id` int(11) NOT NULL auto_increment,
  `time` int(10) default NULL, // the insertion time in unixtimestamp format
  `unique` varchar(255) collate utf8_unicode_ci default NULL, // the website unique file id
  `site` varchar(50) collate utf8_unicode_ci default NULL, // the website unique name
  `url` varchar(255) collate utf8_unicode_ci default NULL, // the file full url
  `filename` varchar(255) collate utf8_unicode_ci default NULL, // the filename, might be blank
  `snippet` varchar(255) collate utf8_unicode_ci default NULL, // the context, usually title, snippet from google search, description, etc
  `tags` varchar(255) collate utf8_unicode_ci default NULL, // tags like "porno", "gays", "hot chicks" etc, usually blank
  `password` varchar(255) collate utf8_unicode_ci default NULL, // if the robot found out there is password, this is where it saves it
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=51048 ;

INSERT queries examples from the latest links:

INSERT INTO `search` VALUES (null, 1225466050, 'WORJGWTK', 'megaupload', 'http://www.megaupload.com/?d=WORJGWTK', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'WORJGWTK', 'megaupload', 'http://www.megaupload.com/?d=WORJGWTK', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'X8JGOPLW', 'megaupload', 'http://www.megaupload.com/?d=X8JGOPLW', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'X8JGOPLW', 'megaupload', 'http://www.megaupload.com/?d=X8JGOPLW', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'W6SPVVBA', 'megaupload', 'http://www.megaupload.com/?d=W6SPVVBA', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'W6SPVVBA', 'megaupload', 'http://www.megaupload.com/?d=W6SPVVBA', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'P8NBWZ3X', 'megaupload', 'http://www.megaupload.com/?d=P8NBWZ3X', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, 'P8NBWZ3X', 'megaupload', 'http://www.megaupload.com/?d=P8NBWZ3X', '', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg');

Code:

INSERT INTO `search` VALUES (null, 1225466050, '156975650', 'rapidshare', 'http://rapidshare.com/files/156975650/sprntl406-notv.part1.rar', 'sprntl406-notv.part1.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466050, '156975660', 'rapidshare', 'http://rapidshare.com/files/156975660/sprntl406-notv.part2.rar', 'sprntl406-notv.part2.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466051, '156975651', 'rapidshare', 'http://rapidshare.com/files/156975651/sprntl406-notv.part3.rar', 'sprntl406-notv.part3.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg'),(null, 1225466051, '156975597', 'rapidshare', 'http://rapidshare.com/files/156975597/sprntl406-notv.part4.rar', 'sprntl406-notv.part4.rar', 'Supernatural S04E06 HDTV XviD-NoTV', '', 'areg');

(the auto increment is kinda old because I fetched this information from an old txt file)


FAQ

Q: Are there any doubled links?
A: There are no double / duplicated links since the robot checks the database for duplicates before adding the links.

Q: Are there any removed (aka no-working) links?
A: There database has NOT been checked for not working links, but the links are from the latest 14 days, so personally, I don't think there are any not-working or removed links.

I even don't know how to quote this thing, so you can just come up with price tags and I may proceed with the sell (;

[Tomer]

מה גורם לך לחשוב שזה חוקי ?