Difference between revisions of "Redmine migration"

From GNU MediaGoblin Wiki
Jump to navigation Jump to search
(initial writeup)
 
(updated migration plan)
Line 18: Line 18:
   
 
= Outstanding issues =
 
= Outstanding issues =
 
== accounts ==
 
 
I can't build accounts for people easily, so I think I'll need to create accounts in Trac and then associate the accounts with the data post-migration. I need to write a script to do the association, but this means there will be a period of time where things will be a little disrupted.
 
 
Is that good enough?
 
 
   
 
== relations ==
 
== relations ==
Line 38: Line 31:
   
 
However, this is a huge pain in the ass to populate after the fact. Is it ok if people re-cc: themselves to bugs they're interested in? If not, I'll spend the time to back-populate the data after the fact, but it'll probably take me a couple of weeks to get to.
 
However, this is a huge pain in the ass to populate after the fact. Is it ok if people re-cc: themselves to bugs they're interested in? If not, I'll spend the time to back-populate the data after the fact, but it'll probably take me a couple of weeks to get to.
  +
  +
 
== text formatting ==
  +
 
Redmine data is in Markdown, but Trac uses its own wiki format. Currently, I'm just passing the text in which makes it look stupid.
  +
 
'''Solution:''' Convert Redmine text data to reST and mark the text with the reST processor. http://trac.edgewall.org/wiki/WikiFormatting#Processors
  +
  +
Code changes need to be made here.
   
   
 
= Solved issues =
 
= Solved issues =
  +
 
== accounts ==
  +
 
I can't build accounts for people easily, so I think I'll need to create accounts in Trac and then associate the accounts with the data post-migration. I need to write a script to do the association, but this means there will be a period of time where things will be a little disrupted.
  +
  +
'''Fixed:''' I wrote a script that trivially allows me to connect new user accounts people create with all their existing bug data.
  +
   
 
== issue numbers ==
 
== issue numbers ==
Line 46: Line 55:
 
Redmine issue numbers don't start with 1 because we're in a Redmine instance with other projects. However, when we move to Trac, tickets will start with 1. So there will be a mismatch between Redmine issue numbers and Trac tickets.
 
Redmine issue numbers don't start with 1 because we're in a Redmine instance with other projects. However, when we move to Trac, tickets will start with 1. So there will be a mismatch between Redmine issue numbers and Trac tickets.
   
'''Fix:''' I'm adding a comment to each Trac ticket that has a link to the original Redmine issue.
+
'''Fixed:''' I'm adding a comment to each Trac ticket that has a link to the original Redmine issue.
   
   
  +
= Things to do after migration =
== text formatting ==
 
   
  +
Here's the list of things we're thinking about doing after we finish the migration and we're using the new system:
Redmine data is in Markdown, but Trac uses its own wiki format. Currently, I'm just passing the text in which makes it look stupid.
 
  +
  +
* create backup scripts for data
  +
* update the Trac front page to reflect MG stuff
  +
* fix workflow to match MG workflow
  +
* modify the reports
  +
* change the style to match other sites we have
   
  +
Anything else?
'''Fix:''' Convert Redmine text data to reST and mark the text with the reST processor. http://trac.edgewall.org/wiki/WikiFormatting#Processors
 
   
   
Line 86: Line 101:
 
** datatotrac - script that loads data in JSON format, downloads attachments, converts it all to Trac tickets, and stores it in the db
 
** datatotrac - script that loads data in JSON format, downloads attachments, converts it all to Trac tickets, and stores it in the db
 
** buildtrac.sh - script that sets up a Trac instance with TracAccountManager plugin on a Debian Stable system
 
** buildtrac.sh - script that sets up a Trac instance with TracAccountManager plugin on a Debian Stable system
  +
** connectuser.py - script to connect new accounts with existing data because i couldn't do it initially
   
 
There were more obstacles than I expected and it just took a long time.
 
There were more obstacles than I expected and it just took a long time.

Revision as of 20:50, 26 January 2012

Redmine Migration

This page covers the impending migration of all our issue data from the Redmine instance at Foocorp to a Trac instance we manage.

This page is a work in progress. I started it so that it can be a one-stop-place for FAQ, schedule, and outstanding issues.


Status

January, 24th 2012: I've gotten the software working. I have Trac installed on http://issues.mediagoblin.org/ with a test migration. Please look around and let me know if you have issues.

Currently working through outstanding issues on the devel mailing list. Once we're done with that, I'll:

  1. remove access to Redmine
  2. do a final migration
  3. update documentation and tell people about our new ticket system


Outstanding issues

relations

Trac doesn't manage relationships between tickets: relates to, blocks, depends, duplicates, ... Trac allows you to resolve a ticket as a duplicate, but doesn't let you specify the ticket that it duplicates. In Trac you do that in the comments when you mark the ticket as a duplicate. My migration script doesn't do anything with the relations information at the moment. I'm open to ideas on what to do here.

Nathan suggested just marking it in a comment on the Trac ticket. Is that good enough?


cc vs watching

Trac has cc section in tickets, but it takes email addresses. Redmine issues have user accounts. Thus in order to connect the two, I need people to create accounts and such.

However, this is a huge pain in the ass to populate after the fact. Is it ok if people re-cc: themselves to bugs they're interested in? If not, I'll spend the time to back-populate the data after the fact, but it'll probably take me a couple of weeks to get to.


text formatting

Redmine data is in Markdown, but Trac uses its own wiki format. Currently, I'm just passing the text in which makes it look stupid.

Solution: Convert Redmine text data to reST and mark the text with the reST processor. http://trac.edgewall.org/wiki/WikiFormatting#Processors

Code changes need to be made here.


Solved issues

accounts

I can't build accounts for people easily, so I think I'll need to create accounts in Trac and then associate the accounts with the data post-migration. I need to write a script to do the association, but this means there will be a period of time where things will be a little disrupted.

Fixed: I wrote a script that trivially allows me to connect new user accounts people create with all their existing bug data.


issue numbers

Redmine issue numbers don't start with 1 because we're in a Redmine instance with other projects. However, when we move to Trac, tickets will start with 1. So there will be a mismatch between Redmine issue numbers and Trac tickets.

Fixed: I'm adding a comment to each Trac ticket that has a link to the original Redmine issue.


Things to do after migration

Here's the list of things we're thinking about doing after we finish the migration and we're using the new system:

  • create backup scripts for data
  • update the Trac front page to reflect MG stuff
  • fix workflow to match MG workflow
  • modify the reports
  • change the style to match other sites we have

Anything else?


FAQ

Why migrate?

Back in July 2011, we created http://bugs.foocorp.net/issues/418 we decided that we needed to manage our own issue tracker rather than use a hosted one. There are various reasons for this, but the biggest one is that we want to be in control of our infrastructure so that we can adapt it to our needs.


Why Trac?

Redmine lacks a tagging system, though I think there are plugins that you can install that handle tagging.

Further, mediagoblin.org has infrastructure for Python web apps. I didn't feel like dealing with Python and Ruby web apps.

I'm more familiar with Trac and can augment it with Trac plugins going forward.


Why 6 months to get this done?

I've spent the 6 months:

  • trying to get Redmine installed and running
  • giving up on that and looking at issue tracking systems
  • fixing bugs in the TracAccountManager plugin
  • figuring out how to set up Trac from source rather than the Debian Stable packages which are Trac 0.11
  • figuring out how to map Redmine to Trac
  • writing a bunch of software to do this:
    • https://github.com/willkg/redminelib - redminelib is a scraping library for Redmine sites
    • redminetodata - script that uses redminelib to download issue data and store it in JSON format
    • datatotrac - script that loads data in JSON format, downloads attachments, converts it all to Trac tickets, and stores it in the db
    • buildtrac.sh - script that sets up a Trac instance with TracAccountManager plugin on a Debian Stable system
    • connectuser.py - script to connect new accounts with existing data because i couldn't do it initially

There were more obstacles than I expected and it just took a long time.