User:Breton/GSoC2014

About me

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:

• jabber: breton@jabber.ru
• email: breton@cynicmansion.ru
• irc: breton in #mediagoblin

Experience

• 3.5 years of Python
• 3 years of Django
• 3.5 years of C in university
• 1.5 years of C++ in university
• 4 years of javascript (though I don't write too much in it); with jQuery and pure
• 6 years of HTML and CSS
• 1.5 year of Scheme and 12 months of Common Lisp
• basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
• a couple of FreeBSD kernel modules

I use

• Debian as main OS
• git (mercurial where required)
• vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

Python 3 port

The goal

Have Mediagoblin running on python3 and python2 with a single codebase.

Rough plan

Requirements

In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

Library	py3?	notes
setuptools	X	Distribute
PasteScript	??	not *technically* necessary;
wtforms	X
py-bcrypt	X
pytest>=2.3.1	X
pytest-xdist	X
werkzeug>=0.7	X
celery>=3.0	X
kombu	X
jinja2	X
sphinx	X
Babel<1.0		Babel 0.9.6 does not support py3
argparse	X
webtest<2	X
ConfigObj	X	officially supported from now
Markdown	X
sqlalchemy>=0.8.0	X
sqlalchemy-migrate		unmaintained anyhow :\
mock	X
itsdangerous	X
pytz	X
six	X
oauthlib==0.5.0	X
unidecode	X

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. TODO.

Version considerations

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian. Though 3.3 has some nice features that would make the migration easier, u strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

Caveats

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

1. Metaclass Syntax Changes -- metaclasses are not used in gmg
2. Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
3. General Iterator Changes -- not used.
4. Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
5. Filesystem and file IO access -- something happens in gmg, not sure yet, what.
6. The last and the biggest -- String Handling

Plan

1. sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
2. adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
3. do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
4. follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
5. write migrations
6. if time left, backport to 3.2

Sort out versions of dependencies

1. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
2. TODO, some stuff didn't work while I was trying to convert to py3.

Start porting code to python3

1. There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
2. http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
3. Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
4. The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
5. To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

Write migrations

• Switch to Alembic
• It still doesn't support hacks around SQLite migrations.
• But there is a workaround that looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

Backport to 3.2

• As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
• Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

What I do and don't know

1. Paste. I have no idea what it is for.
2. sqlalchemy-migrate. Never used it. Though I have worked with django's south.
3. No experience in migrating to Python 3 and no proper experience in Python 3.

Also see /Melange