NetGore

Let me finish it first.

Fine fine I just wanted to see if it worked thanks for working on it though ^^

HURRY SPODI GET IT ALL DONE NOW OR I LEAVE AGAIN. Jk

I'll write a detailed wiki page on it when its done (feel free to remind me if I forget to do so!), but the jest of it is:

Update manager:
- Lets you set the live version or create a new version
- Add new servers in the master or file server lists (more servers just means more sources that are used to download from, thus higher reliability and better performance... also means you can offload some traffic to your home connection if you want)
- Auto-synchronizes both the master and file servers with the live version and next version (if available)

Live version: The current version. This is what the client downloads when it is updating.
Next version: What will be the live version when you move to the next version. Client does not download it, and the next version is always immediately after the live version. Doesn't always exist. When it does exist, it will be synchronized to the servers. This gives you time to upload the new version before making the end-users download the new version.

Master server: A very light-weight, but very critical server that holds the current version number, the list of master & file servers available, and the file information.
File server: Just holds the raw files to download.

The client-side process goes as follows:
1. Use local list of master servers to find the available master servers.
2. Download the remote server listing from each master server available (to update server list), and download live version number.
3. For newly found master servers in the downloaded server list, also check those for the live version number.
4. Use the newest live version number if multiple numbers found (so only one master server must be up-to-date - outdated servers don't break anything).
5. If downloaded live version number == local version number, finish.
6. Download the file listing from the master server for the live version.
7. Find all local files which do not match the hash for the live version.
8. For all of those non-matching files, queue for download from any of the file servers (will rotate between servers when it fails, and update will fail if files cannot be found on any servers after trying each file server at least once for the file).
9. Replace local files with the downloaded ones.
10. For each file that is NOT in the file list, and is NOT in the list of files to skip deleting (which you define per version when creating the version), delete those files. File deletion skip filters are to be used for stuff like configuration files or settings, which you (usually) do not update, but also do not want to delete.

Looks VERY promising at the least

I'd say, once all files are downloaded, have it check their hash again so no files were downloaded but got corrupt because of a hiccup in the ftp or something. So it would just recheck the stuff to make sure everything is OK. Just a suggestion

As for the master server, I'm guessing I'll keep that on my server/garage pc where the Game server is running. I'll use the skyon webhost and the hosting you gave me on your webhost as the two file servers.

My home connection up speed is limited to 150 KB/s while the skyon webhost goes up to 5 to 10 MB/s so I'll probably just stick to those two.

It'll check the hash when the download of a single file finishes (instead of checking all at once at the end), not because of the download protocol (which is actually HTTP by default), but in case any of the files on the server are corrupt. That is also why it forces a rotation through the available servers.

Only real problem I have with the system is that it downloads on a per-file basis. So for a huge update, you could be doing a HTTP download for thousands of files, all uncompressed. But that shouldn't be too big of a problem, and simplifies things a lot (such as being able to easily resume if the client crashes, dealing with slower servers, better distribution, etc). If it is ever a problem, I can change it in the future. No need to over-complicate things from the start.

Yeah, but I can see big updates taking a while though, might wanna add simultaneous file transfers (from the different servers) in the long run. The grh's are a TON of files so if one has a lot of new graphics it'll easily go into the hundreds of png files. But nothing to worry about in the next couple of months.

And yeah, checking after downloading one file sounds fine too. What I'm wondering now though, what's the weird GlobalSettings file 'getting' a bat file for that doesn't seem to be in the sln. And how do you run a master server? I can see a fileserver is just a webhost but I'm guessing the master server will be an executable?

Already done. That is the point of the whole load-balancing thing. Each server also allows a few concurrent downloads, too.

Pretty much all of the files/classes are strange, poorly designed, and unintuitive. That is the problem with programming so fast and not refactoring very much. But I didn't really care since its not like it is going to be expanded on significantly, and nobody really needs to modify anything internally for the most part.

That batch file is for replacing files that were downloaded, but couldn't replace the existing file since it was in use or something. For other OSes, I'll obviously have to add another method than a batch file, but that can be done easily enough.

Its just a web server, or if you add an IDownloadSource for something else, like FTP, then an FTP server.

File servers and master servers are the exact same in implementation, but differ in purpose. They can be hosted on the exact same server using the exact same account information if you want... and that is probably what most people will do.

I've been working on an updater for tanar that downloads the files all (as many as the server/bandwidth allows) at the same time. It's amazing.

Wow Spodi, this really sounds awesome, to such a point you even might want to release it as a stand alone solution for other uses than netgore!

Was thinking that, but was also thinking how much work that'd be required to fix up the code. Then I was like "meh, NetGore already takes up more time than I have."

It might lure a lot more people to the netgore site, aka more exposure for you ^^ but on the other hand, let's not make having it as a stand alone thing a priority atm, there's far more important things for netgore to be done

EDIT:

Yo Spodi can you post in the commit log when you think it's "ready"? As soon as you're satisfied with it, we'll be using it for our SVO Preview Releases.

*Double post *

I'm wondering, is the updater good enough to be used now? Maybe a small explanation ? :3
Also is it possible to update the updater? I'm not sure but this

is kinda the same?

[[GoreUpdater]]

And yeah, should be good enough to use.

Great, we'll probably add it to our next release then (which should be this Sunday).

EDIT: Just read the wiki page, sounds awesome. Just a few quick questions:

\live_version: The current live version.
\master_servers: An up-to-date list of the available master servers.
\file_servers: An up-to-date list of the available file servers.

the master & file servers directory, shouldn't they just be txt files with server IPs?
If I understand correctly, the Update Manager can upload files to the master & file servers?

I also like the idea of running it before the actual client, that way updates always go through and people can't tinker around with files (e.g. texture renaming so they can look like somebody else in their version, ...)

They are just text files, though not guaranteed to remain that way forever. It'd be nice to compress them in the future.

The updater actually doesn't validate all files before running, it just checks hashes to see what needs to be updated when an update is detected (live version != local version). Its not in the slightest bit an anti-cheat, not intended to be, and never will be. People can easily just open the local version file and change their version to some insanely high number to never need updates again. But protecting against stuff like that never really serves any benefit - people will always find a way around it.