OFPEC Forum

OFPEC The Editing Center => Comments & Suggestions => Topic started by: kju on 19 Aug 2009, 12:47:14

Title: CSS - Community sign server
Post by: kju on 19 Aug 2009, 12:47:14

As always not sure if thats the right place.  :D
As OFPEC might be interested to help out on the concept, this might be the right place after all.  :scratch:

Abstract

To avoid addon abuse in Multiplayer one needs to have the signature system (http://community.bistudio.com/wiki/ArmA:_Addon_Signatures) active.

Currently every addon creator is meant to create his key and sign files to be
used by the players and server admins.

Instead an automated system to create the necessary files and provide them
to the players and admins could make everyone's life much easier.


Why?

• Addon makers do sign their addons at times.
• The process is complicated and annoying.
• One mass download and upload source for end users.
• No need to mess with the files. Just press dl all/update.
• Only need to add this one additional Modfolder with zillion of bisign files on the client.

Basic concept

• There is a server or set of server one can send any pbo to.
• The upload process would be best to open to anyone. If abuse is a problem, probably restricted ability to a given group of people might be the alternative.
• The server automatically looks into his database if the addon was already processed.
• If not the server generates a new key file based on the pbo name and signs the pbo with it.
• The pbo file is dropped. So this server does NOT act as a pbo download server.
• Instead the server ONLY provides the key and sign files.
• Now there is a client application to download all sign files in a very simple process. That is all for the client. He only needs to sync with the server before every game session (in which he wants to use addons).
• The server administrators have an access to the CSS to easily filter, select and download or sync the key files of their choice with their server.

Summary

• Client: Can easily download all sign files for any addon out there (and signed by the CSS). As sign files are very small, there is no problem to download thousands or more of em.
  
• Server: Admins have a central place to look for key files and easy access.
  
• Addon creator: There is no longer the need to worry about the sign process. Only needs to submit his work to the CSS for every release.
  

Challenges

• The CSS server. The server hardware to host the system.
• As sign and key files are very small, the download traffic should be minimal.
• The upload interface could be a bottleneck and abuse needs to be ruled out.
• The sign process is simple, however it needs CPU power. That said the
• number of PBOs is limited. So mainly spikes in CPU use, not overall huge load.
• Group of people to code the upload interface.
• Group of people to code the server admin interface to find, filter, select and
• download the key files of choice.
• For the client Yoma Addon Sync or Sickboy's git updater are already available.
• Group of people to code the backend processing
• * filter out already known PBOs via a hash check
• * dump the information about known pbos into a DB (hash, name, date, sign and key file)
• * interaction to upload interface and admin key file interface

Additional thoughts

• Both pbo has and file name could be used to find PBO files at community download sites.
• This system does NOT handle the problem of addons needed by a server or mission. This is a separate problem. Tools are in development to allow this.

FAQ

To be asked first.


Thoughts, feedback and participation in the CSS project is very welcome!



Edit:

Possible implementation details

Design wise I think it is wise to start simple and still keep possible extensions in mind to a degree.

Pretext:
A very simple client side app could improve the system in the way to create a hash client side and
ask the server if the PBO was already signed.
If not, it could 7z the file before uploading to safe traffic / time. This could replace the web-front-end
altogether.


For the front-end indeed a simple web system should do the job:

1) Upload one pbo. Select the file located on your HDD.
2) Not sure if possible to do a client side hash generation. The hash in general is to compare if the file
was already uploaded. If not possible to do client side, the upload probably always has to happen.
Or do you have ideas - the name itself isn't sufficient.
3) If not done, generate a hash on the server for the pbo.
4) Compare the generated hash with all hashes, saved in a SQLiteDB / some DB of choice.
5) If already available, drop PBO and notify the uploader about the existing key.
6) The key is named just like the PBO along with a date tag. I think the day itself is enough like:
myPbo_2009_20_08
7) If has is not found, the system calls DSUtils\DSCreateKey.exe creates the key based on the said schema.
8 ) (Might be done later - just to get the idea) Move the created public key in a location accessible
to the common user. A location available via FTP might be the simplest solution.

This part can be extended though. Meaning either HTTP, or a web-front end to search and filter for
available keys.

For the location probably it might be a good to have .\bikey\datetag\myPbo_2009_20_08.bikey
Maybe the datetag folder level is unnecessary though.

9) We now use the created private key to sign the PBO:
Usage: dsSignFile private_key_filename file_to_sign_filename

This creates a bisign file in addition to PBO. Very small that is. Just a few bytes (306).

10) Check if the creation was successful.
Usage: dsCheckSignatures checked_dir keys_dir

So it is wise to have the PBO, its bisign file alone in a temporary folder during this process to avoid
complexity if another PBO is the reason for the app to fail.

If PBO, bisign and public key fit, the app will return nothing. On error it will report a message.

11) If successful (should be always the case unless PBO is corrupt?), drop the private key.
It is no longer needed and only used once to sign this one very PBO.

12) Create a hash for each PBO, sign and public key, put the hash, their filename and maybe more
information in the DB.

13) Depending on the server, drop the PBO or keep it available for download.
End user download probably would be way too huge traffic. It could serve as a master server for
other servers to mirror it though.

This part might be for a second/later phase to expand.
At least an API to get the hash of PBO for the respective key and/or sign file is possible.
This way an addon download server could query this one to get the key and sign file for the PBO.

14) Move the sign file to a folder with date tag, like .\bisign\2009-08-20\myPbo_2009_20_08.bisign.

15) At this point either have six installer or YAS create the file repository they need to deliver the files.
Both tools allow a very simple install and update system for the end user.

As stated the end user should always only download and update all. As the sign files are so small,
there is no problem to even download thousands of these.

Done.

The core element is to make the admins job easy to look through the available key files for him
to decide, find the one(s) he wants, and easily transfer them to his pc or maybe even server.
The key files are also very small. xxx bytes.