User:Breton/GSoC2014: Difference between revisions

From GNU MediaGoblin Wiki
Jump to navigation Jump to search
(application unstyled)
 
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.
My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.


You can contact me via:
Experience:
* jabber: breton@jabber.ru
- 3.5 years of Python
* email: breton@cynicmansion.ru
- 3 years of Django
* irc: breton in #mediagoblin
- 3.5 years of C in university
- 1.5 years of C++ in university
- 4 years of javascript (though I don't write too much in it); with jQuery and pure
- 6 years of HTML and CSS
- 1.5 year of Scheme and 12 months of Common Lisp
- basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
- a couple of FreeBSD kernel modules


===Experience===
I use:
* 3.5 years of Python
- Debian as main OS
* 3 years of Django
- git (mercurial where required)
* 3.5 years of C in university
- vim
* 1.5 years of C++ in university
* 4 years of javascript (though I don't write too much in it); with jQuery and pure
* 6 years of HTML and CSS
* 1.5 year of Scheme and 12 months of Common Lisp
* basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
* a couple of FreeBSD kernel modules

===I use===
* Debian as main OS
* git (mercurial where required)
* vim


I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start
I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start


Python 3 port.
==Python 3 port==


===The goal===
The goal is to have Mediagoblin running on python3 and a /HackingHowto for those who want it too.


Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):
| Library | py3? | notes |
|--------------------+------+-------------------------------------|
| setuptools | X | Distribute |
| PasteScript | ?? | not *technically* necessary; |
| wtforms | X | |
| py-bcrypt | X | |
| pytest>=2.3.1 | X | |
| pytest-xdist | X | |
| werkzeug>=0.7 | X | |
| celery>=3.0 | X | |
| kombu | X | |
| jinja2 | X | |
| sphinx | X | |
| Babel<1.0 | | Babel 0.9.6 does not support py3 |
| argparse | X | |
| webtest<2 | X | |
| ConfigObj | X | officially supported from now |
| Markdown | X | |
| sqlalchemy>=0.8.0 | X | |
| sqlalchemy-migrate | | unmaintained anyhow :\ |
| mock | X | |
| itsdangerous | X | |
| pytz | X | |
| six | X | |
| oauthlib==0.5.0 | X | |
| unidecode | X | |


{| class="wikitable"
sqlalchemy-migrate is suggested to change to Alembic, which runs python3. TODO: tests with sqlite
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}


sqlalchemy-migrate is suggested to change to Alembic.
Something need to be done with Babel. TODO.

Something need to be done with Babel. '''TODO'''.

====Version considerations====


We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Line 59: Line 102:


The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.
The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====


Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:
Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:


1. Metaclass Syntax Changes -- metaclasses are not used in gmg
# Metaclass Syntax Changes -- metaclasses are not used in gmg
2. Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
3. General Iterator Changes -- not used.
# General Iterator Changes -- not used.
4. Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
5. Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
6. The last and the biggest -- String Handling
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2


=====Sort out versions of dependencies=====
Plan.
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.


=====Adopt tox=====
0. sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
1. adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
2. do something about Paste. The whole paste, as well as pastescript is python3-incompatible.
3. follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
4. write migrations
5. if time left, backport to 3.2


=====Do something about Paste=====
Sort out versions of dependencies.
1. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
X. TODO, some stuff didn't work while I was trying to convert to py3.


=====Start porting code to python3=====
Adopt tox.
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.


=====Write migrations=====
Do something about Paste
* Switch to Alembic
* It still doesn't support hacks around SQLite migrations.
* But there is a workaround that looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite


=====Backport to 3.2=====
Start porting code to python3 generally.
* As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
1. There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
* Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?
2. http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
3. Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
4. The conversion will look like this: 3.3 -> 2.7 -> 2.6 -> 3.2 (-> 3.4?).
5. To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.


==What I do and don't know==
Write migrations.
# Paste. I have no idea what it is for.
Maybe switch to Alembic. It still doesn't support hacks around SQLite migrations.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.
But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite
# No experience in migrating to Python 3 and no proper experience in Python 3.


Also see [[/Melange]]
Backport to 3.2.
As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

Latest revision as of 19:00, 6 April 2014

About me

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:

  • jabber: breton@jabber.ru
  • email: breton@cynicmansion.ru
  • irc: breton in #mediagoblin

Experience

  • 3.5 years of Python
  • 3 years of Django
  • 3.5 years of C in university
  • 1.5 years of C++ in university
  • 4 years of javascript (though I don't write too much in it); with jQuery and pure
  • 6 years of HTML and CSS
  • 1.5 year of Scheme and 12 months of Common Lisp
  • basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
  • a couple of FreeBSD kernel modules

I use

  • Debian as main OS
  • git (mercurial where required)
  • vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

Python 3 port

The goal

Have Mediagoblin running on python3 and python2 with a single codebase.

Rough plan

Requirements

In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

Library py3? notes
setuptools X Distribute
PasteScript ?? not *technically* necessary;
wtforms X
py-bcrypt X
pytest>=2.3.1 X
pytest-xdist X
werkzeug>=0.7 X
celery>=3.0 X
kombu X
jinja2 X
sphinx X
Babel<1.0 Babel 0.9.6 does not support py3
argparse X
webtest<2 X
ConfigObj X officially supported from now
Markdown X
sqlalchemy>=0.8.0 X
sqlalchemy-migrate unmaintained anyhow :\
mock X
itsdangerous X
pytz X
six X
oauthlib==0.5.0 X
unidecode X

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. TODO.

Version considerations

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian. Though 3.3 has some nice features that would make the migration easier, u strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

Caveats

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

  1. Metaclass Syntax Changes -- metaclasses are not used in gmg
  2. Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
  3. General Iterator Changes -- not used.
  4. Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
  5. Filesystem and file IO access -- something happens in gmg, not sure yet, what.
  6. The last and the biggest -- String Handling

Plan

  1. sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
  2. adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
  3. do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
  4. follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
  5. write migrations
  6. if time left, backport to 3.2
Sort out versions of dependencies
  1. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
  2. TODO, some stuff didn't work while I was trying to convert to py3.
Adopt tox
Do something about Paste
Start porting code to python3
  1. There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
  2. http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
  3. Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
  4. The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
  5. To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.
Write migrations
Backport to 3.2
  • As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
  • Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

What I do and don't know

  1. Paste. I have no idea what it is for.
  2. sqlalchemy-migrate. Never used it. Though I have worked with django's south.
  3. No experience in migrating to Python 3 and no proper experience in Python 3.

Also see /Melange

Retrieved from "https://wiki.mediagoblin.org/index.php?title=User:Breton/GSoC2014&oldid=1523"