Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • documentation/doc
  • PhilH/doc
  • louis.csn/doc
  • fabien4vo/doc
  • rngadam/doc
  • anastasia/doc
6 results
Show changes
Showing
with 76 additions and 1204 deletions
.. _reference:
Widgets API Reference
======================
=====================
Widgets are tools that allow to choose, precise how to display data.
Widgets are created at the moment you ask for it for the first time.
They are created at the moment you ask for it for the first time.
The name of the widget is analyzed to build a widget which uses the features and template you want.
Here is a schema of how it works:
......@@ -10,22 +13,31 @@ Here is a schema of how it works:
.. figure:: ../../_static/images/import_documentation/Widgets-API-Developers.png
:alt: Widget API 1
A widget is built like this.
``solid-[type]-[template]-[features]``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*All the keywords must be separated by ``-`` but can be put in any order.*
Its name must begin by "**solid-**", followed by at least a ``type`` (except for ``default`` type) and a ``template``, and eventually one or several ``features``.
*All the keywords must be separated by an hyphen``-``, it is recommended to keep that order (to harmonise widget names writing).*
``type``
++++++++
Can be ``display`` , ``form`` or ``set``. Will select the directory of template to use.
It refers to the "kind" of data to display :
- ``display`` refers to a display formatting,
- ``form`` to display a form element (``<input>``, ``<textarea>``, ``<select>``, etc),
- ``set`` to precise how to display values of fields in :ref:`sets <widget-mixin>`.
It will select the directory of template to use.
``template``
++++++++++++
Name of the template to use.
Each type has several templates. They specify how to use or display the value(s) and/or to indicate the HTML tag to use.
Name of the template to use. Available templates:
Available templates :
- ``display`` directory
......@@ -41,6 +53,7 @@ Name of the template to use. Available templates:
- ``textarea``: set ``value`` in a ``<textarea>``.
- ``richtext``: set ``value`` in an ``<div>`` and initialize a light rich text editor by using ``Quill``.
- ``checkbox``: create a checkbox, which is checked if ``value == true``.
- ``email``: set ``value`` in an ``<input>`` of type ``email``.
- ``date``: set ``value`` in an ``<input>`` of type ``date``.
- ``rangedate``: set 2 values in 2 ``<input>`` of type ``date``. Used for filtering with min and max.
Possible to add ``start-value-[field]`` and ``end-value-[field]`` attributes to set min and max values. Date format to follow : "YYYY-MM-DD". Each attribute accepts ``today`` as value.
......@@ -68,17 +81,17 @@ Name of the template to use. Available templates:
- ``div``: inserts widgets in a ``<div>`` tag.
- ``ul``: inserts widgets in a ``<ul>`` tag.
- **default** (no template keyword defined)
- **default** (no type keyword defined)
- ``action``: Displays a link inside a ``<solid-link>`` tag with ``src`` for the ``data-src`` attribute,
``value`` for the ``next`` attribute and ``label`` as text content.
``value`` for the ``next`` attribute and ``link-text`` as text content.
- ``multiple``: insert a ``solid-display`` with the ``@id`` of a container as ``data-src``.
The fields to display are chosen with the ``multiple-[field name]-fields`` attribute.
``features``
++++++++++++
Features to add to the widget. You can choose as many as you want
They allows to modify the behavior or the display of the widget. It is possible to add one or several features to the widget's name :
- ``label``: adds a ``label`` before the template. Uses the attribute ``name`` if ``label`` is not defined.
- ``labellast``: adds a ``label`` after the template. Uses the attribute ``name`` if ``label`` is not defined.
......
.. _about-our-components:
About our components
######################
####################
**When you should create a component**
......
Convention about commit messages for Startin’blox
=================================================
Commit messages are used to automatically define the version bump of the
next release. Therefore they must reflect the nature of the modification
and the stability of the code.
The commit message should have this pattern: ``type: message``
With the possible commit types:
=========== ========= ================= ========= ============
Commit Type Title Description Release In changelog
=========== ========= ================= ========= ============
``minor`` Minor Minor release ``minor`` ``yes``
``major`` Major Major release ``major`` ``yes``
``feature`` Features New behavior ``patch`` ``yes``
``update`` Updates Change behavior ``patch`` ``yes``
``bugfix`` Bug Fixes Fix bugs ``patch`` ``yes``
``ui`` Interface Change appearance ``patch`` ``yes``
``syntax`` Syntax Refactoring code ``patch`` ``yes``
\ Other All others ``patch`` ``no``
=========== ========= ================= ========= ============
`See
reference <https://git.happy-dev.fr/startinblox/management#commit-messages>`__
Deploy a SIB application on alwaysdata
======================================
- Public: Startinblox dev and devops
- Requires: ``tech@startinblox.com`` permissions
Create account
--------------
1. Create the ``<app>`` account on the chosen server
2. Configure SSH access to the server
Configure a SIB server
----------------------
Installation
~~~~~~~~~~~~
1. From webadmin create a new ``<app>_db`` postgres database
2. Setup python default version to ``v3.6``
3. Configure a ``Python`` website:
- address: ``api.<app>.startinblox.com``
- type: ``Python WSGI``
- path: ``/sibserver/server/wsgi.py``
- workdir: ``/sibserver``
- python version: ``3.6.x``
- statics: ``/static/=/static/``
4. Connect through SSH to `Install a SIB
server <./install_sib_server>`__ with a **production** configuration
::
site_url: https://api.<app>.startinblox.com
allowed_hosts:
- api.<app>.startinblox.com
db_host: postgresql-<app>.alwaysdata.net
db_name: <app>_db
db_user: <app>
db_pass: <db pass>
smtp_host: smtp-<app>.alwaysdata.net
smtp_user: <app>@alwaysdata.net
smtp_pass: <email pass>
admin_email: admin@example.org
admin_name: admin
admin_pass: admin
xmpp_url: 'https://jabber.happy-dev.fr'
jabber_host: 'happy-dev.fr'
Note: Don’t forget to restart app after changing configuration.
Test the setup
~~~~~~~~~~~~~~
SMTP:
::
$ python sibserver/manage.py sendtestemail plup@plup.io
Configure the front application
-------------------------------
1. `Build your application <./build_sib_application>`__ with your
``https://api.<app>.startinblox.com`` URL
2. Transfer the ``dist`` folder content to a ``front`` folder on the
server
3. Configure a ``Static`` website:
- name: Front
- addresses: ``<app>.startinblox.com``
- type: Static files
- root folder: ``/front/``
Develop a new SIB application
=============================
FIXME: complete this section
Develop SIB components
======================
FIXME: complete this section
Develop SIB server packages
===========================
Requirements
------------
- A `SIB server <./install_sib_server>`__ in development mode
Create your package
-------------------
From the project directory (the folder where the ``manage.py`` resides)
create a new package structure:
::
$ sib startpackage mypack
Add your package to the ``packages.yml``:
::
ldppackages:
mypack: mypack
Then initiate your server backend:
::
$ sib install sib_server
Note: It is important to run this command from the project directory in
order for the manager to find your package.
Finally, you can run your server as usual to see your new package’s
data.
Startin’Blox operational documentation
======================================
``SIB`` application capabilities relies on:
- Servers with ``LDP packages`` able to serve the data
- Clients with ``components`` able to manipulate the data
The SIB servers
---------------
Before deploying any SIB applications you will need SIB servers able to
provide all capabilities your application relies on.
SIB servers come with packages developed and `maintained by the
Startin’Blox
team <https://git.happy-dev.fr/startinblox/djangoldp-packages>`__. You
can pick up the ones you like or write your own.
Install a SIB server
~~~~~~~~~~~~~~~~~~~~
You can use any servers exposing data as LDP containers. But you will
likely setup your own to expose your data to others.
- `How to setup a SIB server <./install_sib_server>`__
Develop your own server packages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LDP packages represent capabilites the server could support. Each may
have it’s own configuration depending of what mission it fulfills. There
is no limit of what a LDP package can do.
- `How to develop LDP packages <./develop_sib_ldp_packages>`__
The SIB applications
--------------------
SIB applications are only HTML code binding together web components to
provide all the functionalities the application requires. Each component
can connect to multiple LDP servers providing the matching
functionality. As for the server packages, some SIB components are
developed and `maintained by the Startin’Blox
team <https://git.happy-dev.fr/startinblox/components>`__ but you can
write your owns.
Create an app with SIB components
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- `How to develop a SIB application <./develop_sib_application>`__
Developing your own components
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When developing a new SIB application you can use existing components or
create your owns.
- `How to develop SIB components <./develop_sib_components>`__
Build an existing SIB application
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The easiest to start with is to use an existing SIB application and
build it with your own configuration.
- `How to build an existing SIB
application <./build_sib_application>`__
SIB with Gitlab CI
------------------
- `How to set up automatic delivery for a SIB
application <./setup_continous_delivery_for_sib_application>`__
- `How to set up automatic delivery for
NPM <./setup_continous_delivery_for_node>`__
- `How to set up automatic delivery for
Pypi <./setup_continous_delivery_for_python>`__
SIB in docker
-------------
- `How to run SIB in docker containers <./sib_in_containers>`__
Contributing
------------
- `Architecture overview <./architecture_overview>`__
- `Convention for commit messages <./commit_message_convention>`__
- `Release workflow <./release_workflow>`__
- How to merge ``alpha`` in ``master`` ?
- How next ``alpha`` release is elected ?
Architecture
~~~~~~~~~~~~
Some general schema of the existing StartinBlox applications
architecture can be found
`here <https://docs.google.com/presentation/d/1iCRGaJpFvZjhjIUnpsn6lRTJJ31ES6n94BNkJygzUwM/edit?usp=sharing>`__.
Devops
==========
.. toctree::
:maxdepth: 2
:caption: Devops:
build_sib_application
commit_message_convention
deploy_sib_application_on_alwaysdata
develop_sib_application
develop_sib_components
develop_sib_ldp_packages
home
install_sib_server
manually_force_pushing_on_regitries
migrate_gitlab
release_workflow
setup_continous_delivery_for_node
setup_continous_delivery_for_python
setup_continous_delivery_for_sib_application
sib_in_containers
tests/node_release_management
\ No newline at end of file
How to install a SIB server
===========================
Requirements
------------
The SIB server requires:
- python 3.6
- postgresql database (for production)
Initiate the server
-------------------
From a fresh environment, get the last version of the ``sib-manager``:
::
$ python -m pip install -U sib-manager
Create a project structure from a production template:
::
$ sib startproject sibserver --production
$ cd sibserver
For a development server remove the ``--production`` flag.
Configure your LDP packages
---------------------------
The configuration of the packages goes in ``packages.yml`` file in the
``ldppackages`` section. Those following ones are given as an example.
Yours depend on what your app does.
::
ldppackages:
djangoldp_circle: djangoldp_circle
djangoldp_account: djangoldp_account
djangoldp_profile: djangoldp_profile
oidc_provider: 'git+https://github.com/jblemee/django-oidc-provider.git@develop'
Details about ``packages.yml`` format are given in the template file.
Configure your server parameters
--------------------------------
The ``packages.yml`` files is also use to provide parameters to the
server itself.
Configure the server parameters in the ``packages.yml``:
::
server:
site_url: 'http://localhost:8000'
admin_email: admin@example.org
admin_name: admin
admin_pass: admin
This configuration works for a local development server but for a
production instance you need to setup a postgresql database and
configure dependent services:
::
server:
site_url: 'https://api.batman.happy-dev.fr'
allowed_hosts:
- api.batman.happy-dev.fr
db_host: postgresql-batman.happy-dev.fr
db_name: batman_db
db_user: batman
db_pass: changeit
smtp_host: smtp-batman.happy-dev.fr
smtp_user: batman
smtp_pass: changeit
admin_email: admin@example.org
admin_name: admin
admin_pass: changeit
xmpp_url: 'https://jabber.happy-dev.fr'
jabber_host: 'happy-dev.fr'
Launch the installation
-----------------------
Install/update the project:
::
$ sib install server
Launch the server
-----------------
Run the server in development:
::
$ python manage.py runserver 0.0.0.0:8000
To *launch the server in production*:
- you have to install the static files with
``python manage.py collectstatic``
- and you have to configure your python server to server the script:
``wsgi.py`` on the URL ``api.batman.happy-dev.fr/``.
Tips & tricks
-------------
Activate debug mode
^^^^^^^^^^^^^^^^^^^
To activate the debug mode (default in development) you can override
manually the ``DEBUG`` variable in the ``settings.py``
Install the server as a subfolder URL
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This method isn’t officially supported and requires to change values in
the core configuration.
To setup the SIB server on a subfolder
``https://batman.happy-dev.fr/api`` you have to manually override the
configuration in the ``settings.py``:
::
BASE_URL = 'https://batman.happy-dev.fr'
SITE_URL = 'https://batman.happy-dev.fr/api'
STATIC_URL = '/api/static/'
and URLs in ``urls.py``:
::
urlpatterns = [
url(r'^api/', include('djangoldp.urls')),
url(r'^api/admin/', admin.site.urls),
]
Note: Alwaysdata static config ``/api/static/=/static/``
Manually force pushing on registries
====================================
**Tested for NPM only**
This a ``emergency`` procedure to force pushing on registries. It’s made
to be used when ``continuous delivery pipeline`` is down.
Requirements: This procedure requires authentication on the registries.
After you made your fix, determine the package version to release
(incremented from the last):
::
$ git pull --tags
$ git tag -l
v2.1.0
v2.2.0
Tag the new version to release first and update remote:
::
$ git tag -a 'v2.1.1'
$ git push --follow-tags
This step is important to keep coherency with the autorelease mecanism.
Edit the ``package.json`` to indicate to the registry the version we are
pushing:
::
$ vi package.json
{
"name": "plup-hello",
"version": "v2.1.1",
[truncated]
Push on regitry:
::
$ export NPM_TOKEN=<token>
$ echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}' > .npmrc
$ npm publish
And then reset the version as it is normally managed automatically:
::
$ git checkout -f
$ rm .npmrc
Gitab migration process
=======================
For LDP package release
-----------------------
- Reference to the `configuration
doc <./setup_continous_delivery_for_python#configure-project-on-gitlab>`__
1. Create the new project on https://git.startinblox.com by importing
the existing one from https://git.happy-dev.fr
2. Check the points 2 to 5 of the configuration doc (this is normaly
imported with the project)
3. Apply the point 6 of the configuration doc (variables aren’t
imported)\*
4. Clone the new project or update your remote in your local copy (ex:
``git remote set-url origin git@git.startinblox.com:djangoldp-packages/djangoldp-project.git && git pull``)
5. Update the ``.gitlab-ci.yml`` according to the configuration doc
6. Archive the project on the old gitlab
Note: (*) values for variables can be found in the original project
Releases workflow
=================
- Public: SIB devs
Overview of the code release flow
---------------------------------
.. figure:: ../../_static/images/import_documentation/devops/docs/code_release_flows.png
:alt: code release flows
code release flows
Delivering applications
-----------------------
The applications delivery works with multiple levels of validation:
- Unit testing is automatically triggered for ``all`` branches
- Integration phase is manually triggered on ``feature/*`` branches
- Acceptance phase starts automattically when pushing/merging on
``master`` branch
- Live deployment is manually executed after acceptance on ``master``
branch
.. figure:: ../../_static/images/import_documentation/devops/docs/deployment_pipeline.png
:alt: deployment pipeline
deployment pipeline
Use the ``Pipelines`` menu to control deployment over test environments:
.. figure:: ../../_static/images/import_documentation/devops/docs/test_environments.png
:alt: Test environments
Test environments
And staging + production environments:
.. figure:: ../../_static/images/import_documentation/devops/docs/staging_environments.png
:alt: Staging environments
Staging environments
You can use the ``Environments`` section to track the versions deployed
across environments.
.. figure:: ../../_static/images/import_documentation/devops/docs/all_environments.png
:alt: Environments overview
Environments overview
Working with alpha branches
---------------------------
How to make a hotfix on stable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Propose a hotfix from ``master`` branch:
::
$ git checkout master
$ git checkout -b hotfix/something
# work work work
$ git commit
$ git push -u origin hotfix/something
Then make a proper ``merge request`` into ``master``.
How to make a new stable release from alpha
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``master`` branch only accepts ``fast forward`` merges. So it is
necessary to prepare the prerelease branch in order to make such merges:
::
$ git pull --all
$ git checkout alpha
$ git merge master
$ git push
Then make a proper ``merge request`` into ``master``.
How to set the next prerelease version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The next prelease version is decided by parsing the commit messages
(like normal release) on the next push. Once the prerelease level is set
it is kept until the next release.
Force a bump (according to the configured release rules):
::
$ git checkout beta
$ git commit --allow-empty -m 'minor: force minor bump for beta'
$ git push
If the first commit pushed doesn’t trigger the wanted bump. You still
can delete all prerelease published tags since last stable on gitlab and
the corresponding publication on NPM registry.
Then you can make a commit with the appropriate message and push again.
Setup Gitlab CI/CD for a Node package
=====================================
- Public: developers
- Reading: 5 minutes
Configure project on gitlab
---------------------------
This step requires ``maintainer`` profile on the project.
1. CHeck the ``autodeploy`` user has a ``maintainer`` profile on the
group or project (``Members``)
2. Protect the ``master`` branch allowing only ``maintainers`` to merge
and push (``Settings > Repository > Protected branches``)
3. Allow ``fast forward merge`` only
(``Settings > General > Merge Requests``)
4. Allow ``merge only if the pipeline succeeds``
(``Settings > General > Merge Requests``)
5. Configure ``pipeline strategy`` to ``clone``
(``Settings > CI/CD > Pipelines``)
6. Configure CI/CD variables for authentication
(``Settings > CI/CD > Environment variables``):
============= ============== ==========
Variable Value Protection
============= ============== ==========
``GL_TOKEN`` ``**********`` protected
``NPM_TOKEN`` ``**********`` protected
============= ============== ==========
\*****: Ask to devops for tokens.
Notes:
- The ``fast forward merges`` are mandatory for **prereleases**.
- The ``merge only if the pipeline succeeds`` only makes sense if there
is **unit testing**.
Configure project in code
-------------------------
Release management
~~~~~~~~~~~~~~~~~~
Configure the code packaging by adding a ``release`` section in
``package.json``:
::
$ vi package.json
{
"name": "<your package>",
"version": "0.0.0",
"repository": {
"type": "git",
"url": "https://git.startinblox.com/<path/to/package>.git"
},
"release": {
"branches": [
"master",
{"name": "alpha", "prerelease": true}
],
"plugins": [
["@semantic-release/commit-analyzer", {
"preset": "angular",
"releaseRules": [
{"type": "major", "release": "major"},
{"type": "minor", "release": "minor"},
{"type": "*", "release": "patch"}
]
}],
"@semantic-release/release-notes-generator",
"@semantic-release/gitlab",
"@semantic-release/npm"
]
}
}
Notes:
- Replace ``<your package>`` by the name of your package.
- The ``version`` will be automatically set at deployement step.
References:
- https://github.com/semantic-release/semantic-release/blob/master/docs/usage/configuration.md#configuration
Pipeline execution
~~~~~~~~~~~~~~~~~~
Configure pipeline execution in repository:
::
---
image: node:11
stages:
- test
- release
test:
stage: test
script:
- echo 'Make your tests here !'
except:
- tags
tags:
- sib
publish:
stage: release
before_script:
- npm install -g semantic-release@v17 @semantic-release/gitlab
script:
- semantic-release
only:
- master
- alpha
tags:
- deploy
IMPORTANT: Don’t forget to replace the ``Make your tests here !`` with
the relevant tests for your project. Not testing your application
automatically completly defeat the concept of continuous delivery !
Push configuration
------------------
Send to remote:
::
$ git add .
$ git commit -m 'feature: configured releases'
$ git tag -a v0.1.0 -m 'initial version'
$ git push --follow-tag
Setup prerelease on existing branch
-----------------------------------
Create an ``alpha`` branch from existing ``dev`` branch:
::
$ git pull --all
$ git checkout dev
$ git checkout -b alpha
$ git merge master
$ git push -u origin alpha
Setup Gitlab CI/CD for a Pip distribution
=========================================
- Public: developers
- Reading: 5 minutes
Configure project on gitlab
---------------------------
This step requires ``maintainer`` profile on the project.
1. Check the ``autodeploy`` user has a ``maintainer`` profile on the
group or project (``Members``)
2. Protect the ``master`` branch allowing only ``maintainers`` to merge
and push (``Settings > Repository > Protected branches``)
3. Allow ``fast forward merge`` only
(``Settings > General > Merge Requests``)
4. Allow ``merge only if the pipeline succeeds``
(``Settings > General > Merge Requests``)
5. Configure ``pipeline strategy`` to ``clone``
(``Settings > CI/CD > Pipelines``)
6. Configure CI/CD variables for authentication
(``Settings > CI/CD > Environment variables``):
================= =============== ==========
Variable Value Protection
================= =============== ==========
``PYPI_USERNAME`` ``startinblox`` protected
``PYPI_PASSWORD`` ``**********`` protected
``GL_TOKEN`` ``**********`` protected
================= =============== ==========
\*****: Ask to devops for tokens.
Notes:
- Python projects doesn’t support **prereleases**.
- The ``merge only if the pipeline succeeds`` only makes sense if there
is **unit testing**.
Configure project in code
-------------------------
Release management
~~~~~~~~~~~~~~~~~~
Add ``__version__`` variable somewhere in ``__init__.py``:
::
__version__ = '0.0.0'
Create a file ``setup.cfg`` at the root of your project, alongside the
directory of your app. Configure code packaging by adding a
``semantic-release`` section:
::
[metadata]
name = <your package>
version = attr: <your package>.__version__
description = StartinBlox package
long_description = file: README.md
long_description_content_type = text/markdown
license = MIT
classifiers =
Programming Language :: Python :: 3
[options]
packages = find:
[semantic_release]
version_source = tag
version_variable = <your package>/__init__.py:__version__
commit_parser = commit_parser.parse
Create a file ``.gitlab-ci.yml`` at the root of the project, alongside
the previous one. Edit it to configure pipeline execution in repository:
::
---
image: python:3.6
stages:
- test
- release
# this stage only make sense if you have unit tests
test:
stage: test
script:
- echo 'Make your tests here !'
except:
- tags
tags:
- test
publish:
stage: release
before_script:
- pip install python-semantic-release~=5.0 sib-commit-parser~=0.3
- git config user.name "${GITLAB_USER_NAME}"
- git config user.email "${GITLAB_USER_EMAIL}"
- git remote set-url origin "https://gitlab-ci-token:${GL_TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
- git fetch --tags
script:
- semantic-release publish
only:
- master
tags:
- deploy
Push configuration
------------------
Send to remote:
::
$ git add .
$ git commit -m 'feature: added gitlab CI'
$ git tag -a v0.1.0 -m 'initial version'
$ git push --follow-tag
Deploy your SIB application
===========================
Requirements
------------
In order to run your application you have to configure your web serveur
to serve a static folder (let say ``www``). The configuration itself
depends on your hosting provider.
Set up SSH authentication on your server
----------------------------------------
Generate a keypair (or use an existing one):
::
$ ssh-keygen -t ecdsa -f gitlabci -N '' -C 'deploy@gitlab'
And copy the public key to your server:
::
$ ssh-copy-id -f -i gitlabci.pub <user>@<server>
Pipeline deployment with Gitlab CI
----------------------------------
Configure the ``APP_CONFIG`` variable in the Gitlab pipeline. It is a
JSON string of the configuration file you can generate with
``jq -c . config.json``.
In a ``.gitlab-ci.yml`` file:
::
image: node
stages:
- deploy
deploy_staging:
stage: deploy
environment: staging
script:
- echo "$APP_CONFIG" > config.json
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' > gitlab.key && chmod 600 gitlab.key
- npm install
- npm run build
- scp -i gitlab.key -o StrictHostKeyChecking=no -r dist/* <user>@<host>:~/www/
only:
- staging
SIB in docker containers
========================
This is not intended to support production running.
Setup your SIB server
---------------------
1. Create your ``packages.yml`` according to the documentation.
2. Create the ``Dockerfile``:
::
FROM python:3.6
ENV PATH="/root/.local/bin:${PATH}"
RUN pip install --user -U sib-manager
WORKDIR /opt
RUN cd /opt/ && sib startproject sib_server
ADD packages.yml /opt/sib_server/packages.yml
RUN cd /opt/sib_server && sib install sib_server
EXPOSE 8000
CMD cd /opt/sib_server && python manage.py runserver 0.0.0.0:8000
3. Build the image:
::
# docker build -t sibserver .
4. Run the container:
::
# docker run --rm -p 127.0.0.1:8000:8000 -d sibserver
Serve your client app
---------------------
Launch a container from within your code folder:
::
# docker run --rm -v $PWD:/code -w /code -u $UID -it -p 127.0.0.1:3000:3000 node npm install && npm run watch
Test version releases with node
===============================
Configure release management
----------------------------
Confiugre semantic release in project:
::
$ vi package.json
{
"name": "siblab",
"version": "0.1.0",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"release": {
"branches": [
"master",
{"name": "alpha", "prerelease": true}
],
"plugins": [
"@semantic-release/commit-analyzer"
]
}
}
Setup with gitlab-CI:
::
$ vi gitlab-ci.yml
---
image: node
stages:
- test
- release
test:
stage: test
script:
- echo 'Make your tests here !'
except:
- master
- tags
publish:
stage: release
before_script:
- npm install -g semantic-release@v17 @semantic-release/gitlab
- git checkout $CI_COMMIT_REF_NAME
script:
- semantic-release
only:
- master
- alpha
Test the commit publication in docker
-------------------------------------
::
$ docker run -v $PWD:/code -w /code --name testnode -it node bash
# npm install -g semantic-release @semantic-release/gitlab
# export GL_URL='https://git.startinblox.com'
# export GL_TOKEN=<token>
# export NPM_TOKEN=<token>
# semantic-release --dry-run
Test the prerelease workflow
----------------------------
Patching on alpha
~~~~~~~~~~~~~~~~~
Make a first patch on alpha:
::
$ git commit -m 'init: initial commit'
$ git tag -a v0.1.0 -m v0.1.0
$ git checkout -b alpha
$ git commit --allow-empty -m 'feat: first commit on alpha1'
$ git push --all --follow-tags
Results for ``master`` pipeline:
::
ℹ Found git tag v0.1.0 associated with version 0.1.0 on branch master
ℹ Analyzing commit: fix: hotfix on master
ℹ The release type for the commit is patch
ℹ Analysis of 1 commits complete: patch release
ℹ The next release version is 0.1.1
✔ Created tag v0.1.1
Results for ``alpha`` pipeline:
::
ℹ Found git tag v0.1.0 associated with version 0.1.0 on branch alpha
ℹ Analyzing commit: feat: first commit on alpha1
ℹ The release type for the commit is minor
ℹ Analysis of 1 commits complete: minor release
ℹ The next release version is 0.2.0-alpha.1
✔ Created tag v0.2.0-alpha.1@alpha
Repository after pipelines executions:
::
$ git log --graph --oneline --all --decorate --date-order
* ae5d4a8 (HEAD -> master, tag: v0.1.1, origin/master) fix: hotfix on master
| * 19e6aa9 (tag: v0.2.0-alpha.1@alpha, origin/alpha, alpha) feat: first commit on alpha1
|/
* 19a5d24 (tag: v0.1.0) init: initial commit
Increment patching:
::
$ git checkout alpha
$ git commit --allow-empty -m 'fix: keep working on alpha'
Results for ``alpha`` pipeline:
::
ℹ Found git tag v0.2.0-alpha.1@alpha associated with version 0.2.0-alpha.1 on branch alpha
ℹ Analyzing commit: fix: keep working on alpha
ℹ The release type for the commit is patch
ℹ Analysis of 1 commits complete: patch release
ℹ The next release version is 0.2.0-alpha.2
✔ Created tag v0.2.0-alpha.2@alpha
Merging alpha in master
~~~~~~~~~~~~~~~~~~~~~~~
Merging ``alpha`` in ``master`` triggers the version bump by analyzing
the new commits from ``alpha`` branch. When pushing again in ``alpha``
branch the new tag ``v0.2.0`` is not in the ``alpha`` history, so it
keeps incrementing from previous release.
To make the version bump available on ``alpha`` there is 2
possibilities:
1. Allow only fast forward merge in ``master``:
::
$ git checkout master
$ git merge --ff-only alpha
fatal: Not possible to fast-forward, aborting.
$ git checkout alpha
$ git merge master
$ git checkout master
$ git merge --ff-only alpha
Updating 70f176b..8563ed1
Fast-forward
**merge** ``master`` on ``alpha`` **before** fast forwarding ``alpha``
in ``master``
2. Merge back in ``alpha``:
::
$ git checkout alpha
$ git merge --ff-only master
Updating 4e3f382..a9e1eed
Fast-forward
**merge back** ``master`` on ``alpha`` **after** merging ``alpha`` in
``master``
Repository after pipelines:
::
$ git log --graph --oneline --all --decorate --date-order
* 96abdd6 (HEAD -> master, tag: v0.3.2, origin/master) fix: hotfix in master
| * d78a2ae (tag: v0.3.2-alpha.1@alpha, origin/alpha, alpha) fix: another fix on alpha
|/
* 8563ed1 (tag: v0.3.1) fix: another fix on alpha <-- in both histories (fast forward)
* 70f176b fix: hotfix on master
| * 17e24c3 (tag: v0.3.1-alpha.1@alpha) fix: another commit on alpha after merging back from master
|/
* a9e1eed (tag: v0.3.0) merge: branch 'alpha' <-- in both histories (merged back)
|\
| * 4e3f382 (tag: v0.2.0-alpha.4@alpha) feat: minor bump in alpha
| * 3674bd4 (tag: v0.2.0-alpha.3@alpha) fix: another fix on alpha
* | 5d513b0 (tag: v0.2.0) merge: branch 'alpha' <-- only in master history
|\ \
| |/
| * 4350b55 (tag: v0.2.0-alpha.2@alpha) fix: keep working on alpha
* | ae5d4a8 (tag: v0.1.1) fix: hotfix on master
| * 19e6aa9 (tag: v0.2.0-alpha.1@alpha) feat: first commit on alpha1
|/
* 19a5d24 (tag: v0.1.0) init: initial commit
- How next ``alpha`` release is set ?
- How to merge ``alpha`` in ``master`` ?
......@@ -96,7 +96,7 @@ Some packages may require some configuration to work properly. It is a good prac
Here you have :
* `a list of all the package available <https://git.startinblox.com/djangoldp-packages/>`__
* `Explanation on how to make one <develop-sib-ldp-packages.html>`__
* `Explanation on how to make one <https://docs.startinblox.com/import_documentation/develop-sib-ldp-packages.html>`__
If the description of the available packages is unclear, open an issue to ask for more description.
......@@ -106,27 +106,33 @@ The Server section
This section contains all parameters to the server itself. All this section is loaded as a Django configuration object to initialize the server.
see https://docs.djangoproject.com/fr/2.2/topics/settings/
see https://docs.djangoproject.com/fr/2.2/topics/settings/ for a detail explanation of all settings
The extra config module
-----------------------
The server parameters can be changed at different levels. They are overriden in the following order:
The DjangoLDP server also load a `settings.py` module when it exists in along with the `settings.yml`.
1. The DjangoLDP core default settings
2. :ref:`The DjangoLDP package settings`
3. The code from :ref:`the extra settings module`
4. The `server` section of the YAML settings file
The DjangoLDP packages capabilities
===================================
Which means a value written in the server section of `settings.yml` will override any parameters given by the core or the djangoldp packages.
Among other things, the package has a special file allowing a package to load settings when the djangoldp server starts. The `djangoldp_settings.py` file can reference custom variables and load extra middlewares (they are added to the ones loaded by the djangoldp server itself).
The `MIDDLEWARE` and `INSTALLED_APPS` parameters make exception to that behavior. Those lists are extended from previous values instead of being rewritten. When duplicated entries are detected in those settings, only the first items present in the list are kept.
The extra settings module
-------------------------
The `settings.yml` server section doesn't support dynamical configuration. When this is required, python code for configuration can also be loaded from a `settings.py` module along with the `settings.yml`.
The DjangoLDP package settings
==============================
The DjangoLDP packages support settings declaration through a special `djangoldp_settings.py` at the module's root. This file is fetched by the djangoldp core at runtime and all parameters are loaded following the settings resolution order. (see :ref:`The server section`)
.. code-block:: python
# cat mypkg/mypkg/djangoldp_settings.py
MIDDLEWARE = []
INSTALLED_APPS = []
MYPACKAGE_VAR = 'MY_DEFAULT_VAR'
As any other packages, local packages needs to be referenced in the project `settings.yml`:
.. code-block:: yml
ldppackages:
- mypkg
.. _djangoldp-guide-intro:
Get started with the DjangoLDP server
#####################################
What is behind the DjangoLDP server ?
=====================================
Our solution is based on `Django <https://www.djangoproject.com/>`__ and named `DjangoLDP <https://git.startinblox.com/djangoldp-packages/djangoldp/>`__, adapted to be compatible with Linked Datas convention.
Startin'Blox is a front framework, based on a server named `DjangoLDP <https://git.startinblox.com/djangoldp-packages/djangoldp/>`__ and based on `Django <https://www.djangoproject.com/>`__.The DjangoLDP server is adapted to be compatible with Linked Data's convention.
As a reminder and as specified in :ref:`Startin'Blox presentation <sib-presentation>`, Startin'Blox is data source agnostic,
i.e. it adapts to data sources that are compatible with the Linked Data Platform conventions.
DjangoLDP server, used in production, is the most operational solution today. However, it is currently still
in development, so there is no guarantee that it is complete.
Also, work in progress on compatibility of Startin'Blox with Virtual Assembly's `SemApps <https://www.virtual-assembly.org/semapps/>`__ and Inrupt's `Community server <https://inrupt.com/Solid-roadmap-preview>`__.
Go to `Server Architecture <server-architecture.html>`_ to know more about it.
......@@ -14,16 +23,33 @@ Requirements
The DjangoLDP server requires:
* python 3.6
* python 3.11
* pip and virtualenv installed
* postgresql database (for production)
* sqlite3 (for development)
* Django 4.2
.. note::
If you're working on many project in Python, we advice you to use `a virtual environement <https://virtualenvwrapper.readthedocs.io/en/latest/>`__.
If you're working on many project in Python, we advice you to use `a virtual environment <https://docs.python.org/3/library/venv.html>`__.
.. note::
`See the documentation to set it with Docker <https://docs.startinblox.com/import_documentation/install-sib-server.html#initiate-the-server-with-docker/>`__.
From a fresh environment, get the last version of DjangoLDP:
Optional: Set your virtual environment
=======================================
If you want to set a virtual environment, you can do it with the following commands:
.. code-block:: bash
cd /path/to/your/project
python -m venv myenv
source myenv/bin/activate
Install DjangoLDP
=================
From the virtual environment, get the last version of DjangoLDP:
.. code-block:: bash
......@@ -45,7 +71,7 @@ Add a package to your `settings.yml` file, let's start with `djangoldp_account`:
.. code-block:: yml
dependencies
dependencies:
- djangoldp-account
ldppackages:
......