Develop Bugs

Downloads and Stuff

I know, I know, I know. I'm not paying, so I shouldn't complain.
However, I think we can agree that rapidshare is the opposite of efficient for ROM distribution.
Since I'm no coder myself, just an end user (with an engineering degree in telecoms), for me to set up a good solution myself doesn't exactly strike home eighter.
However, we have some VERY capable coders here. And we have a lot of end users with some bandwidth to share. Is there a chance we can set up an rss-based downloader (using bittorrent to distribute roms), and then simply tell regular downloaders to get the torrent file? (And thus save this site bandwidth and storage, and end users gray hairs over downloads that are SLOOOOOW). Setting up a Diamond "rom-farm" spread over the internet would solve a lot of problems. I know _I_ can prolly share half of my 8mbit (on average), and I think there are others out there who can share aswell, giving this group a good potential (and if we can get other groups to cooperate we can prolly increase bandwidth and spread over the internet).
Bittorrent is ideally suited to this work, especially if we have a few "always on" seeding-servers.
//Svein

This is a very good idea.
I'm on a 100MBit, so that won't be a problem.
I also have a spare dual opteron server lying around.
I've seen that tom (who has a Latvian hosting) also has some good bandwidth.

adwinp said:
This is a very good idea.
I'm on a 100MBit, so that won't be a problem.
I also have a spare dualopteron server lying around.
I've seen that tom (who has a Latvian hosting) also has some good bandwidth.
Click to expand...
Click to collapse
unlimited

What we basically need:
1. Someone with a *nix box on a low-latency (bandwidth isn't an issue, latency is) link, willing to run "our" tracker, and set up the needed accounts for uploads, and set up the tracker to rss-feed the seeder servers.
2. Several people willing to run seeder servers. These need bandwidth. The seeder servers should run in rss-fed full automatic mode.
From what I've seen (between the lines) in the posts on this forum, getting people willing to give bandwidth shouldn't be a problem. What we need is a decent tracker software (I'm open to any suggestions), and some template for setting up the seeder servers.
Am I missing anything? Who volunteers to make the tracker? (I can volunteer to run it, and to make sure latency is ... acceptable on the link. I'm a pedantic millisecond-centric ex netadmin. Ex because of health.)
//Svein

this is a great idea. If i can help in any way I will.

The dual opteron box I was talking about is running gentoo linux.
I will have to reconfig it though.
I can take care of most of the setup during the week-end (if I stay home don't go on the potential trip I planned).
Let's take care of all the suggestions via this thread / pm.

What a great idea

sskogen said:
What we basically need:
Am I missing anything? Who volunteers to make the tracker? (I can volunteer to run it, and to make sure latency is ... acceptable on the link. I'm a pedantic millisecond-centric ex netadmin. Ex because of health.)
//Svein
Click to expand...
Click to collapse
I was a net admin for 4 years so I know how you feel.
^.^
Try testing adwinp.org and see if the latency suits your needs (it should; we are on a 10GBit backbone).
^.^

Well this seems promising! While this whole thing gets in place, as a sort of alpha tryout, how about using public trackers such as thepiratebay for example? And have a wiki page with up to date torrent links.

wow great idea =)
only suggestion here is maybe change the title of this thread xD

I uploaded torrents for the radios and Elite ROMs.
Links are available on the Diamond Radio wiki: http://wiki.xda-developers.com/index.php?pagename=Diamond_ExtractedRadioRoms
and the Diamond Elite ROMs wiki: http://wiki.xda-developers.com/index.php?pagename=Diamond_Elite_ROMs
It should work.. normally... and be faster than rapidshare once we get seeders.
Time to flash Elite 3.0 now
If you already have the files, don't download them again! Just copy them to the torrent download directory, like that your torrent will be 100% complete and you will be able to seed and help others download faster.

Have added the 5 torrents to my local box.
//Svein

Torrent sharing is great. I'll help!

Offer to Cookers

All this rapid share, yadda yadda is really getting old quick. If at least a few ROM Cookers are interested. I'll setup a space on my server to allow uploads that point back to an HTTP download section. You would be responsible for uploading the file, then posting a link here on XDA that would be a direct download link. The uploads would be handled by a passworded FTP account. I have about 400gig of download to spare per month, which should be more than enough for a bunch of Roms. Either reply here or send me a PM and we can talk.
I can also set it up, so that people can go to a root folder and find all the Roms and just pick the ones they want. Heck I can even store older Roms on the server so if someone wants an old one it will be up there.
1st point, there will be no warez posted.
2nd point, you must be a recognized cooker.
Well Cookers, drop a line and let me know.
It would end up being something like this, with each cooker having his own folder.
http://captfiero.com/videos

This has been done, before, twice.
And both times, the file disappeared, or the person did, within 4 days.

Well, not Jannaa's!!!! I use it aplace4u.biz and it's AMAZING!!

veyka said:
This has been done, before, twice.
And both times, the file disappeared, or the person did, within 4 days.
Click to expand...
Click to collapse
The only reason I could see a person leaving is if Microsoft sent a letter saying they did not like people cooking the roms and throwing a hissy fit. I have went ahead and created the folder on my remote server and am uploading stuff right now. I have not made the ftp account for uploading, however I should be able to allow open access to what I have on there in about 2hrs.

Capt Fiero said:
The only reason I could see a person leaving is if Microsoft sent a letter saying they did not like people cooking the roms and throwing a hissy fit. I have went ahead and created the folder on my remote server and am uploading stuff right now. I have not made the ftp account for uploading, however I should be able to allow open access to what I have on there in about 2hrs.
Click to expand...
Click to collapse
I hate rapidshare as much as the next non premium person, so i hope this gets off the ground. people don't even use aspupload, or whatever its called..

Ok the Basic is up, http://www.captfiero.com/romsYou will have to mouse over the file name if it is too long to show up, and then just look at your tool bar to see the entire name.
If anyone see's a file that should not be there. (copyrighted) Please let me know and I will remove it.
Also if a file is something "YOU" made or prefer me not to have up, let me know and I will remove it.
I will not make the upload address public so if you want files uploaded, you can send me a PM and I will upload it, or if you are an established cooker, I will give you direct upload settings.

hmm
I am sure my chef would love this feature, his connection sucks back home though, so if this is genuine I will start mirroring his stuff here too ... I guess the more mirrors the better, I been mirroring to 4shared and mediafire all this time... seems to work well
Just FYI tho, CRCinAU has at least around 100GB of traffic a day (may be he can correct me on this) ...

zocster said:
I am sure my chef would love this feature, his connection sucks back home though, so if this is genuine I will start mirroring his stuff here too ... I guess the more mirrors the better, I been mirroring to 4shared and mediafire all this time... seems to work well
Just FYI tho, CRCinAU has at least around 100GB of traffic a day (may be he can correct me on this) ...
Click to expand...
Click to collapse
He said something like that somewhere.. If you think, most rom topics get 1k+ hits per day.. and say 100 download.. a 50 meg file, that's 5gig from nowhere..
And when a new rom is out..

Well I have 1200gig total traffic allowed, I use about 400gig a month for Videos and I want to keep about 400gig aside for spikes. That leaves me about 400gig to play with. I can't mirror the entire PPC world, but the latest roms and a few add ons would be fine.

heh - where abouts is this mirror located?
At the moment, my mirroring system is handling the load quite well - I have a total of 5 mirrors around the globe (1 x EU, 3 x US, 1 x AU) and the amount of traffic is much lower than it was a month ago on the one AU server because of the way things are spread out now.
The mirroring scripts will auto-detect where the user is and send them to the nearest server (or if we don't have a server near you, we randomly pick one to share the load).
You'll notice this is why I don't have direct links to anything anymore - it's all to my download script.
Even DeviceUpdate uses this script to custom-generate it's index based on where you are
That being said, as of my typing this, since 28th August, I've served:
Release 12.11 ROMs: 270 @ 40Mb each (10.8Gb)
Release 11.2 ROMs: 38 @ 41Mb each (1.55Gb)
Device Update Sessions: 1,235 (unknown download amount)
That's pretty good for less than 4 days
Some more mirrors would be nice - although mirrors in the US are probably not as helpful as the rest of the world... There's ~1Gb of content to be mirrored and count on ~100Gb per month of traffic in a lazy month.

FYI: Developers thread options: http://forum.xda-developers.com/showthread.php?t=388819

To my knowledge it is a US based server. I am personally in Canada however it seems that US hosts are far cheaper. I'll send ya a PM with the details.

File hosting for XDA-Developers.

I have been "leeching" from XDA for a while now and I want to give something back to XDA.
I have a webhosting business. I am not really doing anything with it at this time. I was thinking about opening it up for XDA members for file hosting.
I HATE RapidShare and other "free" sites like it... I wouldnt do anything more than some banner ads on the site, maybe some GoogleAdSense... Nothing intrusive... I promise...
Would anyone use this COMPLETELY FREE SERVICE?
Im setting it up now, 100MB for RAR and ZIP, 50MB for CABS
www.FarBeyondHosting.com

That is a very nice idea! It would be great for this great site.
I join ya! - I absoluty HATE Rapidshare too, most of the time it's busy or I'll keep on countin' the seconds
The adds are no problem, when there aren't a lot - that is...

Most of the rom's are above 100mb though...
WC

Yep file limits may not be the best but its a great idea... thanks for sharing

I have to admit, I dont know what kind of files sizes are needed for the ROMS.
I have used the roms for the 8525, and the tilt, and now the Raphael/Fuze... And those are generally less than 100MB...
What would a good size be? 200? I have unlimited space on that server...

Thanks a lot
I think this is a good idea!

btw has anybody thought how many roms there are published here?

Lol, I was just coming to post the same topic, I have unlimited space and bandwidth if I need it, do you have any bandwidth limits?
I've started hosting files for the android/kaiser project but haven't moved out into the rest of the site so far.
My idea was to just register under a username (hopefully associated with your xda account) and upload your files without size limit (unless it's over 1GB?) and also provide a link for users to view all your posted files/ROMs etc.
Seeing you're already looking into this, maybe I'll still do it, maybe I won't :-D
BTW, thanks for giving back!

Well i have my own physical server in a UK datacenter. Physical as in if i wanted to i could go there and pick it up etc.
It's an intel xeon quad core server with 4gb ram and loads of space!
It's hosted on a dedicated 100mbit connection, with offsite backups located in the USA.
Those intrested ping fbdn.net we run ESXI so basically emulating OS's for better performance etc
I dont see a problem with hosting these files will check my datacenters TOS and AUP in a sec

Torrents

Hi there.
Wouldn't it be nice if xda-developers ha a system for letting the users create torrents when they need to upload something? Many of the developers here use sites like rapidshare and similar to upload their stuff. I don't like those sites and would rather see that the uploaders used a torrent system.
I can create a separate torrent page for you if you like...

Great Idea
dongrif said:
Hi there.
Wouldn't it be nice if xda-developers ha a system for letting the users create torrents when they need to upload something? Many of the developers here use sites like rapidshare and similar to upload their stuff. I don't like those sites and would rather see that the uploaders used a torrent system.
I can create a separate torrent page for you if you like...
Click to expand...
Click to collapse
You nailed it buddy..We need exactly the same !..I am using a 16 MBPS connection and it can't do jack since it seems the ISP is using some proxy kinda system and whenever I try and download...it says your Quota is finished.. Anyways, I hope we have such a system too..I won't mind uploading ROMS using Torrents to other users..

Niice Idea

would be pretty cool and lessen the need for mirrors

Torrents are associated with warez so I don´t think this idea will be moving forwards

We do the association with loading "sinful things" using torrents...so WE as in the general Internet Users started it..I feel that if we start using torrents for sharing ROMs etc..that perception..at least between XDA users will change..and how about Private Trackers...
Explanation as per one of the websites..Looks legit enough to me..
A private tracker is a tracker which allows only registered users to download torrents. The registered users are required to maintain a certain upload-to-download ratio and are banned if the ratio is too low. This ensures that the users don’t just leech the files but also seed them after downloading it completely, which in turn ensures that you get very fast download speeds.

bamit99 said:
We do the association with loading "sinful things" using torrents...so WE as in the general Internet Users started it..I feel that if we start using torrents for sharing ROMs etc..that perception..at least between XDA users will change..and how about Private Trackers...
Explanation as per one of the websites..Looks legit enough to me..
A private tracker is a tracker which allows only registered users to download torrents. The registered users are required to maintain a certain upload-to-download ratio and are banned if the ratio is too low. This ensures that the users don’t just leech the files but also seed them after downloading it completely, which in turn ensures that you get very fast download speeds.
Click to expand...
Click to collapse
Sounds good to me

I've experimented with torrents as a distribution means, but it just doesn't work, as a huge %age of our users were in the UK, most with appalling upload speeds.
You can't expect people to seed a file to a 1:1 ratio if they are on a 30 KBps upload (absolute max), but 8 Mbps down... Bear in mind that at 30 KBps you would have your uploads saturated, so 10 KB/s is more realistic...
And XDA-Developers does not get involved in the distribution of ROMs, or their storage.

Hmm... To be fair my upload speed that I get is about 3kb/s where as download is about 200kb/s
So I won't be able to keep up that ratio

MacaronyMax said:
Hmm... To be fair my upload speed that I get is about 3kb/s where as download is about 200kb/s
So I won't be able to keep up that ratio
Click to expand...
Click to collapse
No worries buddy.. I am sure out of 3 millions XDA Users, we will find a lot with a decent bandwidth Internet...

orb3000 said:
Torrents are associated with warez so I don´t think this idea will be moving forwards
Click to expand...
Click to collapse
Thats very inaccurate. Its just a means of acquiring "illegal" content. Thats like saying cars are associated with hit and runs so nobody should drive.

Ok you words are more precise

nvm, someone beat me to the torrent thing.
Anyway, if torrents were also web-seeded I see no reason why it wouldn't work.

AdrianK said:
nvm, someone beat me to the torrent thing.
Anyway, if torrents were also web-seeded I see no reason why it wouldn't work.
Click to expand...
Click to collapse
Who would pay for the servers required for web seeding?
XDA won't get involved in distribution of ROMs, and as a server admin, I can assure you that I wouldn't want the traffic from a web seeding box on any of my servers... While it's all a lovely hypothetical idea, torrents just don't seem to be the best way to distribute ROMs. Someone ultimately has to upload the file to other peers at their broadband upload speed... Who's gonna wait an hour downloading a file from me, only to then have to seed to someone else? Starting them off would take ages

***Off topic***
Just wondering, never really been in the WM scene, and I was wondering how much roughlyin MB are the Roms?? For android they re 90MB on average, You could get like a stock cynageon rom only being 60mb and a full sense rom 110 ish lol
Thanks

Off topic reply
WM Energy ROM for Leo is about 170Mb but leaves you more free RAM than Android Froyo 2.2...

orb3000 said:
WM Energy ROM for Leo is about 170Mb but leaves you more free RAM than Android Froyo 2.2...
Click to expand...
Click to collapse
Cool thanks

Im thinking I may take this project on, If your a knowledgeable person when ity comes to roms or ideally a dev and would like to be staff on a xda releases torrent site drop me a pm.
If there is allot of interest in this I may provide the site/server/webseeds

I know this is old news but I thought I could at least put my two cents in:
I have been on other private trackers that are closer to a .8:1 ratio. To help those with severely throttled uploads, couldn't we make it more like .1:1?
I am not a coder, but love what you guys do, and I use many of your solutions. I would love to have a way to contribute back on a consistent basis. I have a 1MB upload (which I realize isn't huge) and lots of HD space, and a computer that stays on 24/7. I figure those of us that have a decent upload will help those that can't share much back.

I'd like to say that I support this fully! One more advantage that needs to be listed would the ability to resume downloads and not need to restart corrupt ones!

androidnexus hosting.

So I have multiple VPS's that are ready for some rom hosting or small mod hosting. Running on different servers at different data centers for reliability and to scale up or down.
I have 10 + tb's of bandwidth on the servers for hosting on which have 250 mbps and 2 1gbps ports... Used to run some servers on mine but wiped them clean as i no longer need to do what i used to do with them. I already host some of the roms for a few people. If you want a main download or a mirror just let me know and I can mirrir to my server and my redundancy one so there is always a download option. Or if you want to put your main downloads with me thats cool to.
I just thought Id throw the option out there to help out people that don't trust the reliability of hosting sites or for some reason want a more personal hosting option.
Just let me know if any of you want to use it.
Just thought I'd be able to contribute a little because I haven't been able to dev at all or even modify my own phone lately...

So I am assuming that this is the same guy b-reezy has been talking to me about that he has been using servers from, and got me an account? I will PM you in a minute.

Yea and I will set it up properly So that it can work right. I'll get a page up for downloads etc.

cmsjr123 said:
Yea and I will set it up properly So that it can work right. I'll get a page up for downloads etc.
Click to expand...
Click to collapse
thank you.