Overview

Files.Warwick is an online file sharing service developed in house by the ITS web team at the University of Warwick. With our new Exchange based email system, we have a 10MB attachment limit on both incoming and outgoing emails. This meant that we needed a solution for sending around large files.

Our initial requirements were simply to be able to somehow send and receive files over 10MB in size both internally and with users outside of the University. Since that requirement has been met, we've built the system out further and added more features.

Features

The core Files.Warwick features are:

• Upload files up to 2GB in size over the web. The upload process gives you a progress bar so that you know how far through an upload you are.
• Zip files can be uploaded and extracted into a file space, keeping their structure.
• All activity on Files.Warwick is encrypted by only running over HTTPS, this includes uploads and downloads.
• Create, delete and rename folders so that you can better manage your files.
• International character support for file and folder names.
  
• Allow unauthenticated users to send files to people at Warwick without revealing email addresses.
  
• Allow Warwick users to send files or whole folders to other people at Warwick or external users and be notified when those files are downloaded.
• Personal file spaces are automatically created for users and they are given a unique address such as https://files.warwick.ac.uk/johnsmith
• Group file spaces can be requested for research groups, departments or projects.
• All file spaces can have additional permissions set by the owner so that other users or groups of users can view, change or manage files in that file space.
• Default quota is 2GB for staff and students, but all users can request quota increases which can then be approved or declined.
• You can SFTP securely into your file space or any file space you have admin rights on and upload, download as you would to any other SFTP server. The single file upload limit over SFTP is 4.5GB (if your file space has enough quota).
• Integrated virus scanning checks every uploaded file.
• File space activity logs viewable online so that users and system admins can track file space usage.

Technical Details

Code

Files.Warwick is a Java (version 5) web application built with the Spring Framework and Hibernate.

For user authentication we use our own Web Sign On implementation, but in theory you can protect it with any such system. We also have integrated our Web Groups system which allows our applications to do permissions based on groups such as everyone in a certain module or all staff in a particular department or just an ad-hoc group of researchers.

Application Server

We use the JBoss application server as the container for the web application. We have two instances running at the same time which are then load balanced by the web server. The whole application is stateless so we can easily lose an instance without the end users noticing.

Web Server

We use Apache 2 as the main web server that sits in front of the application server and performs URL mapping to give us nice web addresses and also works with HAProxy to do the load balancing.

Database

All of the application data is stored in an Oracle 10g database. This contains all file metadata, but not the actual files themselves; they are stored on disk.

File System

All files and folders are stored on our NetApp storage system which gives us a very fast and reliable file system. When files are uploaded or downloaded, they are served through the Java webapp rather than straight off the file system as the Java application keeps the permissions information, not the file system itself.

FTP Server

We have a Java FTP(S) server that allows us to offer an FTP(S) interface onto the file spaces. This is based on the Apache FTP Server and then heavily customised to talk to our Java backend rather than straight to the file system. This means we have the exact same security, logging and virus checking for FTP that we have for web uploads.