Magnetico: A Personal Torrent Search Engine That Can’t Be Shut Down

[TRACKERFANCLUB]

The centralized nature of torrent sites means that they're always vulnerable to being shut down. However, a new project called Magnetico aims to solve that problem by crawling BitTorrent's Distributed Hash Table and generating an index on a machine controlled by the user. This week, TF caught up with its creator.

With BitTorrent more than a decade and a half old, it’s one of the most enduring P2P protocols around. It hasn’t been developed much in recent years but there are people out there hoping to take it to the next level.

Much effort has been expended trying to completely decentralize the system. DHT and PEX, for example, were great contributions but the reliance on centralized websites (such as The Pirate Bay) persists.

That particular problem intrigues Bora, a programmer from Istanbul, Turkey, who informs TF that having a less centralized system would be beneficial to the ecosystem as a whole.

“All the legal action taken against the community has always targeted the points of centralization, as they were the weakest points in the whole mesh,” Bora says.

“In the beginning, trackers were required to facilitate the traffic between peers, but then we ditched them once more and more clients supported the distributed hash table (DHT). We eventually got rid of .torrent files as well [in favor of magnet links], which in turn allowed people to download the whole sum of The Pirate Bay in a hundred megabytes.”

In an effort to try and edge closer to the goal of complete decentralization, Bora has been working on a project called magnetico (small ‘m’), which aims to “unplug” conventional torrent sites altogether. Like the recently defunct torrent site BTDigg and the more recent AlphaReign, Bora’s software uses BitTorrent’s DHT to find content and those all-important peers.

However, where magentico differs from the two examples above is that the indexes it creates can be completely private. It effectively acts as a personal torrent search engine that a user can install on a machine under his or her control.

“magnetico is a collection of programs called magneticod (magnetico daemon) and magneticow (magnetico web),” Bora explains.

“magneticod runs in the background and ‘trawls’ the DHT network to discover info hashes and then fetches torrent metadata from the peers (most importantly, the title and the file list). magneticow is a lightweight web interface for users to search and view the discovered torrents.”

 

 

For those interested in some brief technical details, here’s how it works.

“The magneticod module [accesses the DHT] and creates a dummy BitTorrent peer to fetch the metadata from the announced peer, and saves it in a database which you can later search and view,” Bora says.

“As info hashes are the SHA-1 hashes of the torrent metadata, we can be sure of the validity of the metadata we got.”

While operation is simple (Bora says that both modules work without user intervention), the software is in its early stages of development so could be affected by bugs. One known issue is that at times, magneticod can be a bit of a resource hog so may need be restarted.

“I am hoping to address all these issues in the next releases, after collecting useful feedback from the community,” Bora says.

In the meantime, Bora hopes that BEP 51 (a BitTorrent Enhancement Proposal concerning DHT) will come to fruition.

“I would consider myself as having succeeded if major client developers decided to push for BEP 51 and if we together make BitTorrent truly decentralised right in the protocol, rather than resorting back to the brute force solutions,” he concludes.

While magnetico is not for the absolute novice, Bora believes that since its strictly written in Python 3, it’s relatively easy to get going with his detailed instructions.

The magnetico project page can be found here (Github)

https://torrentfreak.com/magnetico-a-personal-torrent-search-engine-that-cant-be-shut-down-170409/