MyUCDavis Course Website Resource Export Tool

Jira: SAK-209

The purpose of this project is to create a tool to allow faculty to migrate their existing course website materials from the MyUCDavis course management system (cms) to the Resources section of the UC Davis implementation of Sakai.

#Background
#Out of Project Scope
#Analysis
Glossary Export
#Design
#Releases - #Release for Links as HTML download
Website Export Screen shots
#Download Stats

Background

IET intends to move faculty and staff using the MyUCDavis cms to using a UC Davis Sakai implementation by Winter 2007. MyUCDavis is a custom-coded, locally-supported ColdFusion application, while Sakai is an open source, collaborative Java application.

To encourage faculty to adopt the new cms, it is imperative that we enable course content migration for both faculty and support staff. Migration must be a smooth and opaque process for faculty. It is also desirable that the migration be consistent with existing functionality in Sakai and be well documented for ease of use by faculty and staff.

Out of Project Scope

The following issues are out of scope of this project:

• Migration of Gradebooks
• Migration of Quizzes
• Migration of Discussion Boards, Chatrooms, Course Email Lists, or Announcements

Analysis

Analysis of Course Website usage data was completed to determine the scope and character of resource content.

Given that Course Website content will be exported in file form, it will be directly compatible with the current Sakai resources tool.

MyUCDavis Course Website Usage Analysis

As of 8/8/2006 there are 10,453 course sites that have resources created in the MyUCDavis Course Website Builder, comprising 221,029 distinct resources. The breakdown of content by type is:

• Folders: 28,239
• Links: 14,758
• Glossaries: 264
• Text Files: 4,765
• Html Files: 5,941
• Uploaded Files: 166,008
  • PDF Files: 83,346
  • Word Documents: 46,907
  • Excel Files: 2,775
  • Powerpoint Files: 13,989
  • Images: 7,098
  • Other Files: 11,893

As of 12/18/06, there are 1685 distinct users in the page_metadata table. The majority represent WebSite Builder users. A list of users from the dev/test system is available for testing on request.

File Size Data Analysis

As of 3/14/07, there are 11,562 course sites listed in the page-metadata table with files of 1Kb or greater stored in Website Builder. Nearly half of the sites in WB contain less than 1 MB of content and two-thirds of sites contain less than 5 MB. Only 6% of sites contain more than 50 MB of content.

Link Data Analysis

As of 3/19/07, there are 13,605 links stored in Website Builder. Almost all of these links are external links - only 5 links have references to my.ucdavis.edu, so the links are exportable and useful to faculty outside the context of MyUCDavis. A breakdown of the database count indicates that the most active usage is within the last two years and during the Fall-Winter-Spring quarters as shown by the top 6 counts (from highest):
Winter Qtr 2007, Winter Qtr 2006, Spring Qtr 2005, Winter Qtr 2005, Fall Qtr 2005, Spring Qtr 2006

*Counts for 2007 are not complete except for Winter quarter since this reflects the future.

Course Management / Sakai Resource Mapping

The mapping of Course Management Website resources to Sakai resources is as follows:

Sakai Import/Export Analysis

There appear to be several methods currently available for a user to import resource content into Sakai:

1. Site Info Tool - Import from Site
   This import process looks for a folder on the server's local disk with the target site id. The folder to be imported is in the Sakai site export process format, which consists of files and xml settings documents.
2. Site Info Tool - Import from File
   This process allows the user to upload a zip file containing archived site content. The file to be imported should consist of a zipped site archive folder, including the requisite xml files.
   NOTE: This feature does not currently work in any of our Sakai environments. Based on Zack Thomas' migration research, "the only archive files this tool understood were those produced by Sakai's proprietary archival functionality and Indiana University's legacy Oncourse system." Use of this tool may require using the Import Service Zack designed.
3. Resource Tool - Add/Add Item(s)
   The resource tool provides a set of file upload fields, providing the ability to upload multiple files to a site's resources area. However, this method requires the user to specifically select each file to be uploaded.
4. Resource Tool - Upload-Download Multiple Resources/WebDAV
   The Sakai webdav service allows users to directly connect to resource sections, allowing for multiple simultaneous file uploads. Additionally, entire folder structures may be imported using this tool.

File Migration from Website Builder to Sakai

The migration strategy from Course Websites into a zip file will follow the following business logic:

1. Files uploaded to Course Websites will be directly packaged for inclusion in the zip output file.
2. Folder structures in Course Websites are virtual, stored only as database entries. Therefore, folder names and structure will be resolved into folder create commands in the zip file.
3. Html and Text pages within the Course Website tools are generated html files which will be added to the zip structure.
4. Links created on the Course Website Home Page and Folder pages will be converted into valid html page representations and stored in index.html files within the appropriate folder.
5. Course Website Glossaries are complex, database-stored sets of elements that do not immediately lend themselves to file export. Therefore, they will not be exported in the initial version.

Analysis Conclusions

Until the Site File Import tool file protocol is determined, the only viable method for import of files into Sakai is via WebDAV. Course Website glossary export will not be implemented in the initial version of resource export.

Design

The course content export process will be made available on a course by course basis. Additionally, the export user will be allowed to choose what files are included in the export.

The migration path will consist of a function in the Course Website Builder to export files from an individual site as a zip file. Faculty and/or staff can then copy the zip file contents to their Sakai course website via webdav. This function will make use of the existing course site copy tool for review of course contents and selection of materials to export.

Desired features of a migration tool are:

1. Faculty, staff and TAs can export and import course website resources
2. The process closely models existing processes, providing for maximum ease of use.

Proposed Course Website Export Process

Website Export Screen shots

1. The user chooses the Website Builder link under their MyUCDavis MyClasses menu options.
2. The user selects the term and course whose content they wish to export.
3. The user chooses the Copy/Export Site link on their Website Builder menu.
4. The user clicks the Export button beneath the Export Instructions text.
5. An interface listing all the course's Website Builder resources is presented.
   • A checkbox will appear next to each exportable element, allowing it to be added to the export.
   • Each unexportable element will be listed, along with the reason.
   • A select/deselect all link will be available.
6. After selecting resources, the user clicks on the "Export to Zip File" Link.
   • The user will be prompted to choose a download location for the file.
7. The user unzips the file to a folder on disk.
8. The user connects to their course resource site via webdav.
   • This may require the user to initiall log into the web interface for sakai to read the directions on webdav and setup the webdav connection.
9. The user copies the extracted folder contents to the mapped webdav directory.
10. Done!

Proposed Website Export/Link Import Process

Initial difficulties were encountered when using WebDAV to import links. Links in Sakai's Resources are not in any recognizable file format and dragging in links is not feasible.

There is a format for Import from File via Site Info, but as noted above, the documentation for implementation via this method is incomplete.

Initially, a method is proposed to allow faculty to enter their links from MyUCDavis by providing them with a list of their link names and the associated url for manual entry into Resources.

Code Modifications

Initial Release

Code modifications to 2 existing files and 3 new files in the directory 'login/CourseManagement/' are required.
Code modifications to 1 existing file and 1 new file in CustomTags is required.
Use of the component zip.cfc in the root directory level 'login' is also required.

Release for Links as HTML download

Updated 10/1/07 to reflect changes in text, addition of link to Smartsite to the file "Adding_Content_to_SmartSite.html", and the addition of the course_links.html file which generates all the users links on a single html page.

Code modifications to 3 existing files in the directory 'login/CourseManagement/' are required.
Code modifications to 1 existing file in CustomTags is required.

Releases

Download Stats

Initial testing of the download zip process by various file sizes provided these stats:

Of the 67% Website Builder users with 5MB or less of content - these users will experience a zip file generation time of 30 seconds or less. The time to download the zip file to the desktop is negligable.

Of the 17% of users with 10-25 MB of content - these users will experience a zip file generation time of 1-2 minutes. Time to download to their desktop is still pretty quick.

Of the 16% of users with 25 MB or more of content - these users will experience anywhere from several minutes to up to 20 minutes (20 minutes for 300 MB) to generate the zip file. The length of time to download is longer and appears to increase exponentially.
For example, it took 7 minutes to generate and then 6 minutes to download 100 MBs of content from one of Caroline Bledsoe's course.