User:Breton/GSoC2014: Difference between revisions
(One intermediate revision by the same user not shown) | |||
Line 141: | Line 141: | ||
* Switch to Alembic |
* Switch to Alembic |
||
* It still doesn't support hacks around SQLite migrations. |
* It still doesn't support hacks around SQLite migrations. |
||
* But there is a workaround |
* But there is a workaround that looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite |
||
=====Backport to 3.2===== |
=====Backport to 3.2===== |
||
Line 151: | Line 151: | ||
# sqlalchemy-migrate. Never used it. Though I have worked with django's south. |
# sqlalchemy-migrate. Never used it. Though I have worked with django's south. |
||
# No experience in migrating to Python 3 and no proper experience in Python 3. |
# No experience in migrating to Python 3 and no proper experience in Python 3. |
||
Also see [[/Melange]] |
Latest revision as of 19:00, 6 April 2014
About me
My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.
You can contact me via:
- jabber: breton@jabber.ru
- email: breton@cynicmansion.ru
- irc: breton in #mediagoblin
Experience
- 3.5 years of Python
- 3 years of Django
- 3.5 years of C in university
- 1.5 years of C++ in university
- 4 years of javascript (though I don't write too much in it); with jQuery and pure
- 6 years of HTML and CSS
- 1.5 year of Scheme and 12 months of Common Lisp
- basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
- a couple of FreeBSD kernel modules
I use
- Debian as main OS
- git (mercurial where required)
- vim
I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start
Python 3 port
The goal
Have Mediagoblin running on python3 and python2 with a single codebase.
Rough plan
Requirements
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):
Library | py3? | notes |
---|---|---|
setuptools | X | Distribute |
PasteScript | ?? | not *technically* necessary; |
wtforms | X | |
py-bcrypt | X | |
pytest>=2.3.1 | X | |
pytest-xdist | X | |
werkzeug>=0.7 | X | |
celery>=3.0 | X | |
kombu | X | |
jinja2 | X | |
sphinx | X | |
Babel<1.0 | Babel 0.9.6 does not support py3 | |
argparse | X | |
webtest<2 | X | |
ConfigObj | X | officially supported from now |
Markdown | X | |
sqlalchemy>=0.8.0 | X | |
sqlalchemy-migrate | unmaintained anyhow :\ | |
mock | X | |
itsdangerous | X | |
pytz | X | |
six | X | |
oauthlib==0.5.0 | X | |
unidecode | X |
sqlalchemy-migrate is suggested to change to Alembic.
Something need to be done with Babel. TODO.
Version considerations
We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian. Though 3.3 has some nice features that would make the migration easier, u strings for example.
The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.
Caveats
Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:
- Metaclass Syntax Changes -- metaclasses are not used in gmg
- Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
- General Iterator Changes -- not used.
- Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
- Filesystem and file IO access -- something happens in gmg, not sure yet, what.
- The last and the biggest -- String Handling
Plan
- sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
- adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
- do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
- follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
- write migrations
- if time left, backport to 3.2
Sort out versions of dependencies
- Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
- TODO, some stuff didn't work while I was trying to convert to py3.
Adopt tox
Do something about Paste
Start porting code to python3
- There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
- http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
- Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
- The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
- To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.
Write migrations
- Switch to Alembic
- It still doesn't support hacks around SQLite migrations.
- But there is a workaround that looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite
Backport to 3.2
- As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
- Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?
What I do and don't know
- Paste. I have no idea what it is for.
- sqlalchemy-migrate. Never used it. Though I have worked with django's south.
- No experience in migrating to Python 3 and no proper experience in Python 3.
Also see /Melange