Here’s a high-level overview of the design I have decided will work best for queueing the recode jobs.
There will be three interacting components.
- The upload server. This is where original contributions reside.
- The queue manager, which can be on any machine or machines running MediaWiki.
- The recode servers.
Here’s a standard use case:
- Video (or audio) upload is received
- Upload server identifies & validates upload as recodable
- Upload server contacts queue manager via http POST, including final url to the file.
- Queue manager replies with an acknowledgement of the job being added and closes the connection. At this point, the upload server can essentially forget about the job and provide upload confirmation to the user.
- The queue manager will track (via mysql for easy shared access among backup queue managers) the status of all recode servers. If there is an idle recode server at the time a job request comes in from the upload server, it is immediately forwarded to that recode server, perhaps with some additional parameters such as output frame size added. The recode manager then monitors periodic reports from the recode server as the job runs, see 6. If there are no idle servers, the job is added to a queue, also implemented as a mysql table.
- Whenever a recode server is given a new job, it keeps the http connection on which the job was given open, and sends status information about the job’s progress over this connection every several seconds. This has two purposes: the main one is to allow the recode manager to detect and handle recode server processes that die unexpectedly, but could also facilitate snazzy ajax reporting of recode status to the user at a later date.
- The recode server starts streaming the original file directly from the upload server via a standard HTTP GET. (Technically, it may be several ranged requests as MPlayer probes the file.) As soon as it has downloaded enough data to generate recoded frames, it opens a POST connection back to the upload server and starts uploading the recoded version. No video data is ever written to disk on the recode server.
- When the entire file has been processed, the recode server notifies the queue manager of success and closes the connection. The queue manager may elect to assign this recode server a new job from the queue table at this time, and processing restarts at 5.
The most interesting exceptional use case is if a recode server fails to recode a file or dies unexpectedly. In the event of a failure message, timed-out or unexpectedly closed connection from the recode server to the queue manager, the queue manager will notify the upload server to delete any recoded data that may have been uploaded, cancel the failed job, and attempt to assign a new job to that recode server. If it cannot, it will update the recode server status table to reflect the failed server.
The child process and connection control routines necessary to do the above can be implemented entirely in PHP. Particularly, the queue manager will not need to be anything more than a PHP script/web service.