This is exactly what I was looking for! A way to keep my games backed up (and up to date) in my local hard drive.

... Now if only I could figure out how to install this HTML5 thing it's complaining about, haha.

https://github.com/html5lib/html5lib-python

Should just be:

$ pip install html5lib

If you don't have pip, install that first. What OS are you using?
Noted. Seems silly given that a single IP could host many different individuals/accounts.

Perhaps they're using a combination of account and IP. I don't know as I have only one account.

I'm only glad they're just blocking the IP (MaGog's) and not the account (mine), so the block only affects MaGog's data collection and not me as a person.

Ah, I finally got it working by using the commandline and doing "setup.py install". I"m using Windows 7 64 by the way.

Now I get another issue though. It is attached.

Sounds like you're using Python 3.0? You need 2.7 (2.6 may also work). I'll add Python 3.0 support in the todo list... I can't imagine there would be lots of work to do there (unless html5lib didn't support Python 3.0 which would surprise me).


Nope.. definitely IP banned. account is fine.. when I'm banned, I run it from a linode box and it works fine (same login credentials).

What is the difference to lgogdownloader?

btw login does not work:
$ gogrepo.py login user password
20:31:15 | attempting gog login as 'user' ...
20:31:16 | fatal...
Traceback (most recent call last):
File "gogrepo.py", line 758, in <module>
main(process_argv(sys.argv))
File "gogrepo.py", line 732, in main
cmd_login(args.username, args.password)
File "gogrepo.py", line 268, in cmd_login
for elm in etree.findall('.//script'):

$ python --version
Python 2.7.3
$ dpkg-query -l | grep html5lib
ii python-html5lib 0.95-1 all HTML parser/tokenizer based on the WHATWG HTML5 specification
$ uname -a
Linux debby 3.2.0-4-686-pae #1 SMP Debian 3.2.65-1+deb7u2 i686 GNU/Linux

I'll try when I get home.. I wonder if they changed the element tree. If so should be an easy fix.

I can't reproduce your issue. I tried Windows, Linux, and FreeBSD... putting wrong info will print an error as expected:

$ ./gogrepo.py login user pass
16:59:27 | attempting gog login as 'user' ...
16:59:29 | login failed, verify your username/password and try again.

Here's a successful login:
$ ./gogrepo.py login
enter username:
enter password:
19:40:33 | attempting gog login as 'xxx@xxx.com' ...
19:40:36 | login successful!


I wonder if GOG is serving you a different page than me. It would help if you could send me the "web_data" data before it is parsed..

in cmd_login(), this is where it parses login page (note that this is before the user / pass is even sent to GOG:

# get the auth_url
info("attempting gog login as '%s' ..." % user)
with request(GOG_HOME_URL, delay=0) as web_data:
etree = html5lib.parse(web_data, namespaceHTMLElements=False)
for elm in etree.findall('.//script'):
if elm.text is not None and 'GalaxyAccounts' in elm.text:
auth_url = elm.text.split("'")[1]
break

>>> import html5lib
>>> html5lib.__version__
u'0.999'

This is a great project and thanks for it, I'll probably use it after I buy a big fat TB drive .. but IMO we should push for gog to include this sort of "backup facility" in Galaxy ... wouldn't this be nice? ;)

Can you specify the download path? Does it create subdirectories for each product? Do the names it creates jive with the existing GOG downloader, or would I end up with a lot of duplication?

Absolutely agree! It would be best if GOG Galaxy user could keep all installers and updates in sync as an official feature.


Yes you can specify download path; just specify it as a parameter to download

ex.
./gogrepo.py download /mnt/gog-games

If not specified, by default it downloads to the current directory the script is run from

Yes, it creates a folder for each game and puts all files and bonus files inside. It also generates a !info.txt and !serial.txt. See the readme for more details.

It uses the same naming conventions as the gog downloader. The directory and file names are the same as GOG Downloader's. I may add a feature to be able to map your own dirs too (if you want to name them your own way rather than GOG's default names).

ex.
shadowrun_returns/
shadowrun_returns/!info.txt
shadowrun_returns/!serial.txt
shadowrun_returns/gog_shadowrun_returns_1.0.0.4.tar.gz
shadowrun_returns/gog_shadowrun_returns_dlc1_1.0.0.1.tar.gz
shadowrun_returns/patch_shadowrun_returns_2.2.1.11.exe
shadowrun_returns/setup_shadowrun_dragonfall_dlc_2.2.0.10.exe
shadowrun_returns/setup_shadowrun_returns_2.2.0.10.exe
shadowrun_returns/shadowrun_returns_avatar.zip
shadowrun_returns/srr_dragonfall_artworks.zip
shadowrun_returns/srr_dragonfall_avatars.zip
shadowrun_returns/srr_dragonfall_poster.zip
shadowrun_returns/srr_dragonfall_renders.zip
shadowrun_returns/srr_dragonfall_wallpaper.zip
shadowrun_returns/srr_wallpaper.zip

If you have an existing GOG collection, I would suggest you do a verify to see how much it recognizes
./gogrepo.py login
./gogrepo.py update
./gogrepo.py verify <your_gogdownloader_gamesdir>

If you have space and want a clean folder, you can do
./gogrepo.py backup <your_gogdownloader_gamesdir> <new_game_dir>

If you have files which do not follow GOG downloader format, then you can still use the import function which will check MD5.. if any matches to your game manifest database, then they'll get copied over !

I'll soon add a "clean" function that can delete extra or "old" GOG files. This can help to keep your collection clean and only have latest files.

Do you handle duplicates? For example, same extras in unbundled games?

Great question. I had thought about this, and decided (for now) to just ignore that fact and let it download the duplicate bonus files, keeping all items self-contained sort of as GOG intends. I'm not really tight for space to worry about dupes.

Also gog doesn't provide MD5 data for most bonus files... in fact I don't even bother trying in order to save web fetch and lessen tempban.

I think this could be improved though if there are some "very wasteful" examples.

On Debian 7 (Wheezy) it's HTLM5lib 0.95 you get at best (I already added it). After uninstalling the package python-html5lib 0.95 and obtaining it via pip install html5lib (1.06b) it is working.

Perhaps you add at the first page (and github page) that you will need at least HTML5lib 0.999 at least :)

Some suggested features (some are fluff):

- have some sort of filter system (regexp)
- store your settings and cache in ~/.config/gogrepo and ~/.cache/gogrepo
- add a switch to optionally set the title of the terminal what it is currently doing
- a switch for log file
- store the extras in gamename/extras like lgogdownloader
- pull the screenshots from the store page
- get the game tile and store it as folder.jpg for the folder icon
- have some sort of orphan check and move files not provided by gog to a backup folder (includes older game releases)

Sorry for stomping on you and having many suggestions :)

See also https://github.com/eddie3/gogrepo/issues/1 :)