Browse Source

Update documentation

master
Joshua Rubingh 8 months ago
parent
commit
699a691b9e
  1. 11
      VRE/apps/university/serializers.py
  2. 2
      VRE/apps/virtual_machine/providers/vrw/serializers.py
  3. BIN
      doc/Database_structure.png
  4. 6
      doc/OpenStack.rst
  5. 12
      doc/VRW.rst
  6. BIN
      doc/_images/datamodel-vre.png
  7. 12
      doc/development.rst
  8. BIN
      doc/documentation.pdf
  9. 12
      doc/models.rst
  10. 4
      doc/requirements.txt
  11. 12
      doc/signals.rst
  12. 2318
      doc/swagger.yaml
  13. 14
      doc/tus.rst

11
VRE/apps/university/serializers.py

@ -25,6 +25,7 @@ class FacultySerializer(ModelSerializer): @@ -25,6 +25,7 @@ class FacultySerializer(ModelSerializer):
first_name (str): :attr:`~user.first_name`
last_name (str): :attr:`~user.last_name`
email_address (str): :attr:`~user.email`
university (object): :attr:`~apps.university.serializers.UniversitySerializer`
"""
university = UniversitySerializer()
@ -35,6 +36,16 @@ class FacultySerializer(ModelSerializer): @@ -35,6 +36,16 @@ class FacultySerializer(ModelSerializer):
class StudyFieldSerializer(ModelSerializer):
"""
This serializer will only pick the fields :attr:`~apps.university.models.StudyField.id` and :attr:`~apps.university.models.StudyField.name` from the :class:`~apps.university.models.StudyField` model.
And it will add the full data of the :attr:`~apps.university.models.Faculty` where this study field belongs to.
Args:
id (int): :attr:`~apps.university.models.University.id`
name (str): :attr:`~apps.university.models.University.name`
faculty (object): :attr:`~apps.university.serializers.FacultySerializer`
"""
faculty = FacultySerializer()

2
VRE/apps/virtual_machine/providers/vrw/serializers.py

@ -88,7 +88,7 @@ class WorkspaceStatusUpdateSerializer(serializers.Serializer): @@ -88,7 +88,7 @@ class WorkspaceStatusUpdateSerializer(serializers.Serializer):
This serializer is used for updating the :term:`VRW` status.
Args:
status (str): The new status for this :term:`VRW` (:class:`apps.vrw.models.WorkspaceStatus`).
status (str): The new status for this :term:`VRW` (:class:`apps.virtual_machine.providers.vrw.models.WorkspaceStatus`).
remote_id (str): The new ID that is created by the remote cloud system
"""

BIN
doc/Database_structure.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

After

Width:  |  Height:  |  Size: 764 KiB

6
doc/OpenStack.rst

@ -6,19 +6,19 @@ This part of the application is for integrating with the OpenStack at HPC for cr @@ -6,19 +6,19 @@ This part of the application is for integrating with the OpenStack at HPC for cr
------
Config
------
.. automodule:: apps.openstack.apps
.. automodule:: apps.virtual_machine.providers.openstack.apps
:members:
------
Models
------
.. automodule:: apps.openstack.models
.. automodule:: apps.virtual_machine.providers.openstack.models
:members:
-------
Signals
-------
.. automodule:: apps.openstack.signals
.. automodule:: apps.virtual_machine.providers.openstack.signals
:members:
--------

12
doc/VRW.rst

@ -9,39 +9,39 @@ Authorization @@ -9,39 +9,39 @@ Authorization
This part of the API is only accessible with accounts that are added to a special :term:`VRW` API group. By default this group is called 'vre-api' and be changed with the setting ':attr:`~settings.VRW_API_GROUP`'. This is done so that normal users are not able to change :term:`VRW` statusses.
The authentication is done by the general REST API.
.. automodule:: apps.vrw.permissions
.. automodule:: apps.virtual_machine.providers.vrw.permissions
:members:
------
Config
------
.. automodule:: apps.vrw.apps
.. automodule:: apps.virtual_machine.providers.vrw.apps
:members:
------
Models
------
.. automodule:: apps.vrw.models
.. automodule:: apps.virtual_machine.providers.vrw.models
:members:
-----
Views
-----
.. automodule:: apps.vrw.views
.. automodule:: apps.virtual_machine.providers.vrw.views
:members:
-----------
Serializers
-----------
.. automodule:: apps.vrw.serializers
.. automodule:: apps.virtual_machine.providers.vrw.serializers
:members:
-------
Signals
-------
.. automodule:: apps.vrw.signals
.. automodule:: apps.virtual_machine.providers.vrw.signals
:members:
--------

BIN
doc/_images/datamodel-vre.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 764 KiB

12
doc/development.rst

@ -139,7 +139,7 @@ After installation of the packages, create a symbolic link in the `/etc/nginx/si @@ -139,7 +139,7 @@ After installation of the packages, create a symbolic link in the `/etc/nginx/si
Important parts of the VHost configuration:
.. literalinclude:: ../nginx/tus.vhost.conf
.. literalinclude:: ../../Upload_Server/nginx/tus.vhost.conf
:language: bash
And there should be a `lua` folder in the `/etc/nginx` folder. This can be a symbolic link to the LUA folder that is provided with this project.
@ -172,12 +172,12 @@ The daemon needs to know the following information. These settings are required: @@ -172,12 +172,12 @@ The daemon needs to know the following information. These settings are required:
This information can be placed in an .env file in the same folder where the startup (*startup.sh*) script is located. An example .env file:
.. literalinclude:: ../tusd/.env.example
.. literalinclude:: ../../Upload_Server/.env.example
:language: bash
In the startup.sh script there are some default variables that can be overwritten by adding them to the .env file above.
.. literalinclude:: ../tusd/startup.sh
.. literalinclude:: ../../Upload_Server/startup.sh
:language: bash
:lines: 5-16
@ -190,7 +190,7 @@ The upload data is stored at a folder that is configured in the TUS startup comm @@ -190,7 +190,7 @@ The upload data is stored at a folder that is configured in the TUS startup comm
Hooks
-----
The TUS is capable of handling hooks based on uploaded files. There are two types of hooks. 'Normal' hooks and webhooks. It is not possible to run both hook systems at the same time due to the blocking nature of the pre-create hook. So we use the 'normal' hook system. That means that custom scripts are run. Those scripts can then post the data to a webserver in order to get a Webhook functionality with the 'normal' hooks.
At the moment, there is only a HTTP webcall done in the hook system. There is no actual file movement yet.
At the moment, there is only a HTTP API call done in the hook system. There is no actual file movement yet.
For now we have used the following hooks:
- **pre-create**: This hook will run when a new upload starts. This will trigger the REST API server to store the upload in the database, and check if the upload is allowed based on an unique upload url and unique upload code.
@ -198,7 +198,7 @@ For now we have used the following hooks: @@ -198,7 +198,7 @@ For now we have used the following hooks:
An example of a hook as used in this project is the *pre-create.py* script.
.. literalinclude:: ../tusd/hooks/pre-create.py
.. literalinclude:: ../../Upload_Server/hooks/pre-create.py
This hook uses the same data payload as when TUS would use the Webhook system. So using 'Normal' hooks or using Webhooks with REST API Server should both work out of the box.
@ -235,5 +235,5 @@ Now you can access your demo portal at http://localhost:8000. In order to login @@ -235,5 +235,5 @@ Now you can access your demo portal at http://localhost:8000. In order to login
There are more settings available to setup. These can be added the to .env file of the demo portal.
.. literalinclude:: ../demo_portal/demo_portal/.env.example
.. literalinclude:: ../../Demo_Portal_1/demo_portal/demo_portal/.env.example
:language: bash

BIN
doc/documentation.pdf

Binary file not shown.

12
doc/models.rst

@ -56,6 +56,18 @@ Virtual Machine @@ -56,6 +56,18 @@ Virtual Machine
.. automodule:: apps.virtual_machine.models
:members:
----------------------------
Virtual Machine Provider VRW
----------------------------
.. automodule:: apps.virtual_machine.providers.vrw.models
:members:
----------------------------------
Virtual Machine Provider OpenStack
----------------------------------
.. automodule:: apps.virtual_machine.providers.openstack.models
:members:
-----
Token
-----

4
doc/requirements.txt

@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
Sphinx==3.5.1
Sphinx==4.2.0
sphinx-markdown-builder==0.5.4
sphinxcontrib-openapi==0.7.0
sphinxcontrib-openapi==0.7.0

12
doc/signals.rst

@ -19,3 +19,15 @@ Researcher @@ -19,3 +19,15 @@ Researcher
----------
.. automodule:: apps.researcher.signals
:members:
----------------------------
Virtual machine provider VRW
----------------------------
.. automodule:: apps.virtual_machine.providers.vrw.signals
:members:
----------------------------------
Virtual machine provider OpenStack
----------------------------------
.. automodule:: apps.virtual_machine.providers.openstack.signals
:members:

2318
doc/swagger.yaml

File diff suppressed because it is too large Load Diff

14
doc/tus.rst

@ -17,7 +17,7 @@ In order to communicate with the REST API server, we need some settings to be en @@ -17,7 +17,7 @@ In order to communicate with the REST API server, we need some settings to be en
* DROPOFF_API_HAWK_KEY
* DROPOFF_API_HAWK_SECRET
.. literalinclude:: ../tusd/.env.example
.. literalinclude:: ../../Upload_Server/.env.example
:language: bash
NGINX / LUA
@ -29,14 +29,14 @@ So basically, NGINX is needed in order to be able to add study information to th @@ -29,14 +29,14 @@ So basically, NGINX is needed in order to be able to add study information to th
**NGINX config:**
.. literalinclude:: ../nginx/tus.vhost.conf
.. literalinclude:: ../../Upload_Server/nginx/tus.vhost.conf
:lines: 54-91
:language: bash
**LUA code:**
**JavaScript code:**
.. literalinclude:: ../nginx/lua/dropoff_tus.lua
:language: lua
.. literalinclude:: ../../Upload_Server/nginx/njs/dropoff_tus.js
:language: javascript
Python hooks
@ -50,7 +50,7 @@ pre-create @@ -50,7 +50,7 @@ pre-create
When a new upload is started, the *pre-create* hook will check if the study ID is valid and that the upload can start. This will also anounce the upload to the REST API server and therefore can be found in the overview of data drops.
.. literalinclude:: ../tusd/hooks/pre-create.py
.. literalinclude:: ../../Upload_Server/hooks/pre-create.py
post-finish
@ -66,6 +66,6 @@ This hook is a bit more complex and does multuple things. @@ -66,6 +66,6 @@ This hook is a bit more complex and does multuple things.
* Report back if upload is processed correctly
.. literalinclude:: ../tusd/hooks/post-finish.py
.. literalinclude:: ../../Upload_Server/hooks/post-finish.py
:linenos:

Loading…
Cancel
Save