...
Sakai
...
Data
...
Storage
...
-
...
Campus
...
Installations
...
Existing
...
File
...
System
...
-
...
AFS
...
Currently,
...
UC
...
Davis
...
people
...
have
...
home
...
directories
...
in
...
AFS
...
space,
...
and
...
that
...
CourseManagement
...
also
...
utilizes
...
this
...
space.
...
- User's
...
- AFS
...
- Space
...
- Allocation
...
User
...
- space
...
- in
...
- AFS
...
- is
...
- in
...
- the
...
- form
...
- of
...
- the
...
- directory
...
- stucture
...
- below:
...
Ex.
...
/afs/home.ucdavis.edu/home/sn/samerson
...
Above,
...
- sn
...
- is
...
- the
...
- first
...
- initial
...
- of
...
- firstname,
...
- and
...
- last
...
- initial
...
- of
...
- lastname.
...
- Course
...
- AFS
...
- Space
...
- Allocation
...
AFS
...
- space
...
- for
...
- courses
...
- is
...
- allocated
...
- via
...
- CRN,
...
- and
...
- CourseManagement
...
- in
...
- MyUCDavis
...
- writes
...
- data
...
- to
...
- this
...
- directory.
...
- The
...
- ACL
...
- is
...
- the
...
- MyUCDavis
...
- user.
...
Ex.
...
/afs/.ucdavis.edu/myucd/class/200603/ARE112001/
...
Above
...
- uses
...
- TermCodeCRN,
...
- and
...
- class
...
- uniqueness
Problem Statement
Here are some of the questions we need to answer:
- How do we integrate Sakai's resources and our current AFS file system?
Answer: via replacing ContentHostingService with our own implementation for AFS storage. The current ContentHostingService, backed by DbContentService provides minimal file storage capability - When user's create worksites, how does this get mapped?
Answer:
For user worksites, those would resolve to their current afs home mapping scheme (e.g. home/sn/samerson)
...
For
...
- other
...
- worksites,
...
- the
...
- /afs/
...
- root
...
- will
...
- be
...
- used
...
- as
...
- a
...
- starting
...
- point
...
- to
...
- store
...
- content.
...
- Project
...
- sites
...
- will
...
- have
...
- 2
...
- types
...
- (user
...
- project
...
- sites,
...
- and
...
- institutional
...
- project
...
- sites),
...
- and
...
- course
...
- sites
...
- will
...
- be
...
- stored
...
- via
...
- /TermCode/CRN/Sakai
...
- instance
...
- structure.
...
- See
...
- AFS
...
- File
...
- Mapping
...
- documentation
...
- for
...
- more
...
- details.
...
- What
...
- location
...
- does
...
- it
...
- get
...
- mapped?
...
Answer:
...
- /afs/
...
- as
...
- root
...
- path
...
- for
...
- now,
...
- to
...
- be
...
- determined
...
- at
...
- a
...
- later
...
- point
...
- when
...
- to
...
- split
...
- this
...
- pathing
...
- up..
...
- Do
...
- course
...
- type
...
- of
...
- worksites
...
- get
...
- stored
...
- in
...
- existing
...
- AFS
...
- CourseManagement
...
- space?
...
Answer:
...
- Yes,
...
- this
...
- will
...
- get
...
- saved
...
- in
...
- the
...
- existing
...
- AFS
...
- structure,
...
- however
...
- each
...
- instance
...
- of
...
- Sakai
...
- (e.g.
...
- SmartSite,
...
- CERE)
...
- will
...
- be
...
- able
...
- to
...
- write
...
- to
...
- the
...
- content
...
- appropriately.
...
- Where
...
- do
...
- project/research/
...
- or
...
- any
...
- other
...
- non-user
...
- and
...
- non-course
...
- Sakai
...
- sites
...
- get
...
- stored?
...
Answer:
...
- under
...
- the
...
- /afs/
...
- root
...
- path,
...
- but
...
- we
...
- need
...
- to
...
- determine
...
- a
...
- volume
...
- and/or
...
- whether
...
- or
...
- not
...
- the
...
- site
...
- name
...
- resolves
...
- to
...
- anything
...
- meaningful
...
- over
...
- time.
...
How Sakai stores content
Current Sakai architecture supports storing course and user content both inside/outside of a database. The content path can be mapped to any given path one gives it in the sakai.properties configuration file:
ref: http://bugs.sakaiproject.org/confluence/display/FAQ/2.2.7.1+Configuration
...
The
...
content
...
below
...
is
...
taken
...
from
...
bugs.sakaiproject.org/confluence:
...
| No Format |
|---|
The best place for configuring this is the sakai.properties file.
# the file system root for content hosting's external stored files (default is null, i.e. store them in the db)
bodyPath@org.sakaiproject.service.legacy.content.ContentHostingService =${sakai.home}content
Enable the above line, and point at the root folder for the files to be stored.
# when storing content hosting's body bits in files, an optional set of folders just within the content.filesystem.root
# to act as volumes to distribute the files among - a comma separate list of folders. If left out, no volumes will be used.
bodyVolumes@org.sakaiproject.service.legacy.content.ContentHostingService = v1,v2,v3
Enable the above line, and set the list of "volumes" for storage. You can specify one or more volume names, comma separated on this line. These are folders under the file system root. Files will be distributed among these volumes.
If you are going to use multiple volume devices, you need to map them to these volume names that live "under" the root. We have done this with our AFS file storage system at the University of Michigan. If you are not using separate devices, then you can use any folder names for the volumes. Provide at least one.
Files will be stored under each volume in a way so that there are not too many in any one folder. The folder structure we use is:
{{YYYY/DDD/HH/id, where YYYY=year, DDD=day of year, HH=hour of day, and the 1111...=an id-based file name}}
for example,
{{2005/070/03/3223479379834-2343}}
or, using the above root and volumes, it might be:
{{/usr/local/tomcat/sakai/content/v2/2005/070/03/3223479379834-2343}}
Note that the resource name and type is not at all encoded here. The date/time used to form the file name is the date/time of file creation.
{noformat}
h3. Proposed |
Proposed solution(s)
...
For
...
both
...
spaces,
...
use
...
the
...
ACL
...
associated
...
with
...
the
...
current
...
MyUCDavis
...
user,
...
however
...
make
...
the
...
reference
...
now
...
the
...
Sakai
...
user.
...
Use
...
the
...
same
...
IP's
...
registered
...
with
...
the
...
MyUCDavis
...
user
...
for
...
the
...
Sakai
...
user.
...
This
...
solution
...
should
...
be
...
targeted
...
for
...
Fall
...
Quarter
...
2006.
...
However,
...
in
...
the
...
meantime
...
a
...
proposed
...
solution
...
is
...
to
...
use
...
/afs/
...
as
...
the
...
root
...
path
...
for
...
file
...
content
...
storage
...
so
...
that
...
a
...
common
...
file
...
system
...
can
...
be
...
utilized.
...
Until
...
the
...
AFS
...
solution
...
can
...
be
...
fully
...
implemented,
...
we
...
are
...
to
...
utilize
...
this
...
strategy.
...
a.
...
user's
...
space
...
Create
...
a
...
.sakai
...
directory
...
within
...
the
...
user's
...
AFS
...
space
...
that
...
the
...
Sakai
...
user
...
account
...
has
...
access
...
to
...
write
...
to.
...
User's
...
will
...
not
...
be
...
allowed
...
to
...
browse
...
this
...
directory,
...
since
...
it
...
is
...
only
...
pertinent
...
to
...
Sakai.
...
b.
...
course
...
space
...
Utilize
...
University
...
of
...
Michigan's
...
current
...
mapping
...
(see
...
above
...
for
...
format
...
of
...
file
...
path)
...
of
...
course
...
content.
...
This
...
will
...
be
...
a
...
new
...
directory
...
structure
...
from
...
what
...
is
...
currently
...
being
...
used
...
for
...
MyUCDavis
...
CourseManagement.
...
c.
...
projects/research
...
space
...
Sakai
...
has
...
many
...
types
...
of
...
sites,
...
and
...
each
...
install
...
can
...
configure
...
these.
...
There
...
will
...
be
...
sites
...
related
...
to
...
projects
...
and
...
research
...
that
...
can
...
be
...
expected.
...
User's
...
space,
...
and
...
possibly
...
another
...
space
...
in
...
AFS,
...
will
...
be
...
utilized
...
for
...
these
...
types
...
of
...
sites.
...
Content
...
stored
...
in
...
user's
...
AFS
...
space
...
will
...
count
...
against
...
the
...
user's
...
AFS
...
quota,
...
specifically
...
the
...
user
...
who
...
owns
...
the
...
site.
...
Implementation details
The default registered service that handles content in Sakai is the DbContentService, which extends the BaseContentHostingService. A modification to the path generation in the DbContentService will allow the path to be configured for proper content storage in AFS. Creating a UCDavisContentService, which extends DbContentService, will enable custom configurations and "overrides" of methods which establish the file path used to store content. There are several considerations for this to be handled properly:
1. The site type must be determined, so the user's space may be used or that of CourseManagement.
This is currently working.
2. The user's quota should be checked prior to storage?
Needs to be implemented
3. The path must be customizable
Needs to be implemented
A preliminary UCDavisDbContentService file is provided, which will replace the DbContentService as the registered service for storing content. This is a Sakai 2.1 example.
Implementation Algorithm In Summary
*Note: The algorithm below refers to file writes into AFS, from Sakai. File read process involves a separate process of reading file paths from the Sakai internal database, and trying to access this file in AFS. Currently, standard Sakai error handling will apply to file read errors.
- Determine bodypath for file storage, from ContentHostingService
- Determine if bodypath exists via ContentHostingService. If exists, try to
- Find site type from a Sakai reference object (that ContentHostingService uses) if site type is:
- User Sites
- Determine userid
- From userid, determine user AFS home root from firstname, lastname algorithm. This is AFS HOME ROOT
- Create bodypath as /afs/home.ucdavis.edu/home/
...
- AFS
...
- HOME
...
- ROOT
...
- /.sakai/
...
- INSTANCE
- Personal Project Sites
- Course Sites
- Other Site Types (e.g.
...
- Institutional
...
- Project
...
- Sites)
...
- else if bodypath doesn't
...
- exist..
...
Things to consider
- There will be a need (tool?)
...
- to
...
- display
...
- how
...
- much
...
- user's
...
- space
...
- is
...
- taken
...
- by
...
- Sakai,
...
- etc.
...
- This
...
- will
...
- reduce
...
- the
...
- amount
...
- of
...
- support
...
- calls,
...
- and
...
- be
...
- consistent
...
- with
...
- the
...
- current
...
- CourseManagement
...
- tools
...
- available
...
- for
...
- MySpace.
...
- The
...
- path
...
- for
...
- storing
...
- content
...
- should
...
- be
...
- highly
...
- configurable
...
- via
...
- properties,
...
- or
...
- other
...
- variables.
...