From 8604d3bc1b096af2621ccb5385b31461cd222807 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 20:57:07 +0100 Subject: [PATCH 1/9] [#99] README.md update --- README.md | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 43d4a4426..9234de035 100644 --- a/README.md +++ b/README.md @@ -4,22 +4,18 @@ Stork is a new project started by ISC in October 2019 with the aim of delivering BIND and Kea dashboard. It is going to be a spiritual successor of earlier attempts - Kittiwake and Antherius. -It is currently in very early stages of planning. More information will become publicly -available in October 2019. Stay tuned! +It is currently in very early stages of development. -# Getting Started - -Please see the Installation section in the Stork ARM for details. Stork ARM will -be published soon, but for the time being you can generate it on your own by -doing this: - -```console -rake doc -``` - -Make sure you have rake and sphinx installed. +For details, please see [Stork Administrator Reference Manual](https://stork.readthedocs.io). # Build instructions The easiest way to run Stork is with docker (`rake docker_up`). However, it is possible to run Stork without docker. See Installation section of the Stork ARM. + +# Getting involved + +Stork is in very early stages of its development. There isn't much available yet that end users +could use. However, if you'd like to get involved, feel free to subscribe to the +[stork-dev mailing list](https://lists.isc.org/mailman/listinfo/stork-dev) or look +at [Stork project page](https://gitlab.isc.org/isc-projects/stork). -- GitLab From 2c7940b5fcb04e9cc2045c4d0f47f5a105069260 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 20:58:07 +0100 Subject: [PATCH 2/9] [#99] User and machines management described --- doc/index.rst | 7 ++-- doc/usage.rst | 109 +++++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 101 insertions(+), 15 deletions(-) diff --git a/doc/index.rst b/doc/index.rst index 8da45aac6..56eff64ea 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -10,10 +10,9 @@ It is going to be a spiritual successor of earlier attempts - Kittiwake and Anth It is currently in very early stages of planning. This is the reference guide for Stork version |release|. -Links to the most up-to-date version of this document (in PDF, HTML, -and plain text formats), along with other documents for -Kea, can be found in ISC's `Stork project homepage `_. - +Links to the most up-to-date version of this document, along with other documents for +Stork, can be found in ISC's `Stork project homepage `_ +or `readthedocs `_ . .. toctree:: :numbered: diff --git a/doc/usage.rst b/doc/usage.rst index cc397a613..d45568e7d 100644 --- a/doc/usage.rst +++ b/doc/usage.rst @@ -4,22 +4,109 @@ Using Stork *********** -Setting up users -================ +This section describes how to use features available in stork. To connect to Stork, use your +web browser and connect to port 4200. If Stork is running on your localhost, you can navigate +to http://localhost:4200. - As of Stork 0.1.0 release, there is no possibility to create new users. This - functionality will be added in the next release. Currently, the default - administrator's account is created and can be used to sign in to the - system via the web UI. Please use the login `admin` and password `admin` to - sign in to the system. +Managing users +============== +Currently, the default administrator's account is created and can be used to sign in to the system +via the web UI. Please use the login ``admin`` and password ``admin`` to sign in to the system. -Monitoring machines with BIND and Kea -===================================== +To manage users, click on the ``Configuration`` menu and choose ``Users``. You will see a list of +existing users. At the very least, there will be user ``admin``. -.. todo:: +To add new user, click ``Create User Account``. A new screen will open that will let you specify the +new account parameters. Some fields have specific restrictions. Username can consist of only +letters, numbers and underscore. E-mail field is optional. However, if specified, it must be a well +formed e-mail. First and lastname fields are mandatory. Password must only contain letters, digits, +@, ., !, +, - and must be at least 8 characters long. Once all requirements are met, the ``Save`` +button will become active and you will be able to add new account. - Describe how the user can deploy agent, connect new machines, how to monitor them, etc. +.. note:: + As of 0.2.0, the role-based access control is not implemented yet. Every user added is + considered a super-admin and is able to do everything the interface allows. +Deploying Stork Agent +===================== + +Stork system uses agents to monitor services. Stork Agent (`STAG` or simply `agent`) is a small +daemon that is expected to be deployed and run on each machine to be monitored. As of 0.2.0 there +are no automated deployment routines and STAG has to be copied and run manually. This can be done in +a variety of way. Here is one of them. + +Assuming you want to monitor services running on machine with IP 192.0.2.1, you can do the following +on the Stork server command line: + +``` +cd +scp backend/cmd/stork-agent login@192.0.2.1:/path +``` + +On the machine to be monitored, you need to start the agent. In the basic case, you can simply +run it: + +``` +./stork-agent +``` + +You can optionally pass ``--host=`` or set the ``STORK_AGENT_ADDRESS`` variable to specify which +address the agent will listen on. You can pass ``--port`` or set the ``STORK_AGENT_PORT`` +variable to specify which TCP port the agent will listen on. + +.. note:: + + Unless explicitly specified, the agent will listen on all addresses on port 8080. There are no + authentication mechanisms implemented in the agent yet. Use with care! + +Registering New Machine +======================= + +Once the agent is deployed and running on the machine to be monitored, you should instruct stork +server to start monitoring it. You can do so by going to Services menu and choosing Machines. +You will be presented with a list of currently registered machines. + +To add a new machine, click ``Add New Machine``. You need to specify the machine address or hostname +and a port. If Stork agent is running in a container, you may specify the container name. This is +particularly useful, if you built stork using ``rake docker_up`` command and the agent is running in +a container. In such case, you can use kea-agent as your hostname. If you run agent by using ``rake +run_agent``, the agent will listen on port 8888. + +Once you click Add, the server will attempt to establish gRPC over http/2 connection to the agent. +Make sure that any firewalls in between will allow incoming connections to the TCP port specified. + +Once a machine is added, number of parameters, such as hostname, address, agent version, number +of CPU cores, CPU load, available total memory, current memory utilization, uptime, OS, platform +family, platform name, OS version, kernel, virtualization details (if any), host ID and other +information will be provided. More information will become available in the future Stork versions. + +Monitoring Machines +=================== + +To monitor registered machines, go to Services menu and click Machines. A list of currently +registered machines will be displayed. Pagination mechanism is available to display larger +number of machines. + +There is a filtering mechanism that acts as an omnibox. The string typed is searched for in address, +agent version, hostname, OS, platform, OS version, kernel version, kernel architecture, +virtualization system, host-id fields. The filtering happens once you hit ENTER. + +You can inspect the state of a machine by clicking its hostname. A new tab will open with machine +details. Multiple tabs can be open at the same time. You can click Refresh state to get updated +information. + +The machine state can also be refreshed using Action menu. On the machines list, each machine has +its own menu. Click on the triple lines button at the right side and choose the Refresh option. + +Deleting Machines +================= + +To stop monitoring a machine, you can go to the Machines list, find the machine you want to stop +monitoring, click on the triple lines button at the right side and choose Delete. Note this will +terminate the connection between Stork server and the agent running on the machine and the server +will no longer monitor it. However, the Stork agent process will continue running. If you want to +completely shut it down, you need to do so manually, e.g. by connecting to the machine using ssh and +stopping the agent there. One way to achieve that is to issue ``killall stork-agent`` command. -- GitLab From d41b5a51592864f78c7b3fe0c518579670421884 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 21:07:23 +0100 Subject: [PATCH 3/9] [#99] Formatting fix --- doc/usage.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/usage.rst b/doc/usage.rst index d45568e7d..1f4e695e3 100644 --- a/doc/usage.rst +++ b/doc/usage.rst @@ -41,17 +41,17 @@ a variety of way. Here is one of them. Assuming you want to monitor services running on machine with IP 192.0.2.1, you can do the following on the Stork server command line: -``` -cd -scp backend/cmd/stork-agent login@192.0.2.1:/path -``` +.. code:: console + + cd + scp backend/cmd/stork-agent login@192.0.2.1:/path On the machine to be monitored, you need to start the agent. In the basic case, you can simply run it: -``` -./stork-agent -``` +.. code:: console + + ./stork-agent You can optionally pass ``--host=`` or set the ``STORK_AGENT_ADDRESS`` variable to specify which address the agent will listen on. You can pass ``--port`` or set the ``STORK_AGENT_PORT`` -- GitLab From 6ecb165cd2969c50bf67de4e71960af8984e8099 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 21:07:36 +0100 Subject: [PATCH 4/9] [#99] Backend API rewritten --- doc/backend-api.rst | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/doc/backend-api.rst b/doc/backend-api.rst index dfe5505a0..690a2127a 100644 --- a/doc/backend-api.rst +++ b/doc/backend-api.rst @@ -1,17 +1,12 @@ .. _backend-api: -***************** -Stork Backend API -***************** +*********** +Backend API +*********** -URL: (stork-url)/version-get - -Returns: - -.. code-block:: json - - { - "version": "1.2.3" - } +Stork agent provides a REST API. The API is generated using [Swagger](https://swagger.io/). The API +points are currently documented in the ``api/swagger.yaml`` file. +.. note:: + In the future Stork releases, the API documentation will be generated automatically. -- GitLab From 25126b34533914c8682bc1d9e8379ef73f2042d2 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 21:18:39 +0100 Subject: [PATCH 5/9] [#99] Agent API documented --- doc/devel.rst | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/doc/devel.rst b/doc/devel.rst index 80bc436ae..d668d8fa4 100644 --- a/doc/devel.rst +++ b/doc/devel.rst @@ -9,3 +9,45 @@ Developer's Guide We acknowledge that users and developers are two different groups of people, so the documents should eventually be separated. However, since these are still very early days of the project, this section is kept in the Stork ARM for convenience only. + +Agent API +========= + +The connection between the server and the agents is done using gRPC over http/2. The agent API +definition is kept in the ``backend/api/agent.proto`` file. For debugging purposes, it is possible +to connect to the agent using [grpcurl](https://github.com/fullstorydev/grpcurl) tool. For example, +you can retrieve a list of currently provided gRPC calls by using this command: + +.. code:: console + + $ grpcurl -plaintext -proto backend/api/agent.proto localhost:8888 describe + agentapi.Agent is a service: + service Agent { + rpc detectServices ( .agentapi.DetectServicesReq ) returns ( .agentapi.DetectServicesRsp ); + rpc getState ( .agentapi.GetStateReq ) returns ( .agentapi.GetStateRsp ); + rpc restartKea ( .agentapi.RestartKeaReq ) returns ( .agentapi.RestartKeaRsp ); + } + +You can also call specific gRPC calls. For example, to get the state, the following command can be +used: + +.. code:: console + + $ grpcurl -plaintext -proto backend/api/agent.proto localhost:8888 agentapi.Agent.getState + { + "agentVersion": "0.1.0", + "hostname": "copernicus", + "cpus": "8", + "cpusLoad": "1.68 1.46 1.28", + "memory": "16", + "usedMemory": "59", + "uptime": "2", + "os": "darwin", + "platform": "darwin", + "platformFamily": "Standalone Workstation", + "platformVersion": "10.14.6", + "kernelVersion": "18.7.0", + "kernelArch": "x86_64", + "hostID": "c41337a1-0ec3-3896-a954-a1f85e849d53" + } + -- GitLab From 4a5345d60a792fc4a2c8abb24f1eeb699a534b30 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 21:20:27 +0100 Subject: [PATCH 6/9] [#99] ChangeLog updated --- ChangeLog.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ChangeLog.md b/ChangeLog.md index ba6366d9d..8c5bc533f 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,3 +1,8 @@ +* 15 [doc] tomek + + Users and machines management is now documented in the Stork ARM. + (Gitlab #99) + * 14 [doc] sgoldlust Introduced new Stork logo in the documentation. -- GitLab From 2babf0faf39c9f6cef2c2ffea97753a5e4be20ca Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 21:30:51 +0100 Subject: [PATCH 7/9] [#99] Doc generation described --- doc/devel.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/doc/devel.rst b/doc/devel.rst index d668d8fa4..c3917099c 100644 --- a/doc/devel.rst +++ b/doc/devel.rst @@ -10,12 +10,18 @@ Developer's Guide should eventually be separated. However, since these are still very early days of the project, this section is kept in the Stork ARM for convenience only. +Generating Documentation +======================== + +To generate documentation, simply type ``rake doc``. You need to have Sphinx and rtd-theme installed. +The documentation will be available in the ``doc/singlehtml`` directory. + Agent API ========= The connection between the server and the agents is done using gRPC over http/2. The agent API definition is kept in the ``backend/api/agent.proto`` file. For debugging purposes, it is possible -to connect to the agent using [grpcurl](https://github.com/fullstorydev/grpcurl) tool. For example, +to connect to the agent using `grpcurl `_ tool. For example, you can retrieve a list of currently provided gRPC calls by using this command: .. code:: console -- GitLab From ca6c987bcbb485a1c3cbc9dee493e51904d8a095 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Tue, 3 Dec 2019 21:44:01 +0100 Subject: [PATCH 8/9] [#99] --host, --port described in stork-agent --- doc/man/stork-agent.8.rst | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/doc/man/stork-agent.8.rst b/doc/man/stork-agent.8.rst index 9d7bcfb60..1385b4a00 100644 --- a/doc/man/stork-agent.8.rst +++ b/doc/man/stork-agent.8.rst @@ -15,7 +15,7 @@ stork-agent - Stork agent that monitors BIND and Kea services Synopsis ~~~~~~~~ -:program:`stork-agent` +:program:`stork-agent` [**--host**] [**--port**] Description ~~~~~~~~~~~ @@ -24,6 +24,22 @@ The ``stork-agent`` is a small tool that is being run on the systems that are running BIND and Kea services. Stork server connects to the stork agent and uses it to monitor services remotely. +Arguments +~~~~~~~~~ + +The Stork Agent takes the following arguments: + +``-h`` or ``--help`` + Displays list of available parameters. + +``--host=hostname`` + Specifies the IP to listen on. Can be controlled with $STORK_AGENT_ADDRESS environment + variable. The default value is ``::``. + +``--port=1234`` + Specifies the TCP port to listen on for connections. The default is 8080. Can be controlled + with $STORK_AGENT_PORT environment variable. + Configuration ~~~~~~~~~~~~~ -- GitLab From 3a5208236addcb1308d17efa9ea0a3fdd1252c35 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Wed, 4 Dec 2019 13:26:38 +0100 Subject: [PATCH 9/9] [#99] Several editorial changes As a result of the review. --- doc/devel.rst | 11 ++++++----- doc/usage.rst | 24 ++++++++++++------------ 2 files changed, 18 insertions(+), 17 deletions(-) diff --git a/doc/devel.rst b/doc/devel.rst index c3917099c..a3e9f86cc 100644 --- a/doc/devel.rst +++ b/doc/devel.rst @@ -13,13 +13,14 @@ Developer's Guide Generating Documentation ======================== -To generate documentation, simply type ``rake doc``. You need to have Sphinx and rtd-theme installed. -The documentation will be available in the ``doc/singlehtml`` directory. +To generate documentation, simply type ``rake doc``. You need to have `Sphinx `_ +and `rtd-theme `_ installed. The generated +documentation will be available in the ``doc/singlehtml`` directory. Agent API ========= -The connection between the server and the agents is done using gRPC over http/2. The agent API +The connection between the server and the agents is established using gRPC over http/2. The agent API definition is kept in the ``backend/api/agent.proto`` file. For debugging purposes, it is possible to connect to the agent using `grpcurl `_ tool. For example, you can retrieve a list of currently provided gRPC calls by using this command: @@ -34,8 +35,8 @@ you can retrieve a list of currently provided gRPC calls by using this command: rpc restartKea ( .agentapi.RestartKeaReq ) returns ( .agentapi.RestartKeaRsp ); } -You can also call specific gRPC calls. For example, to get the state, the following command can be -used: +You can also make specific gRPC calls. For example, to get the machine state, the following command +can be used: .. code:: console diff --git a/doc/usage.rst b/doc/usage.rst index 1f4e695e3..49280d765 100644 --- a/doc/usage.rst +++ b/doc/usage.rst @@ -17,7 +17,7 @@ via the web UI. Please use the login ``admin`` and password ``admin`` to sign in To manage users, click on the ``Configuration`` menu and choose ``Users``. You will see a list of existing users. At the very least, there will be user ``admin``. -To add new user, click ``Create User Account``. A new screen will open that will let you specify the +To add new user, click ``Create User Account``. A new tab will opened that will let you specify the new account parameters. Some fields have specific restrictions. Username can consist of only letters, numbers and underscore. E-mail field is optional. However, if specified, it must be a well formed e-mail. First and lastname fields are mandatory. Password must only contain letters, digits, @@ -26,17 +26,17 @@ button will become active and you will be able to add new account. .. note:: - As of 0.2.0, the role-based access control is not implemented yet. Every user added is - considered a super-admin and is able to do everything the interface allows. + As of Stork 0.2.0 release, the role-based access control is not implemented yet. Every user + is considered a super-admin and has full control over the system. Deploying Stork Agent ===================== -Stork system uses agents to monitor services. Stork Agent (`STAG` or simply `agent`) is a small -daemon that is expected to be deployed and run on each machine to be monitored. As of 0.2.0 there -are no automated deployment routines and STAG has to be copied and run manually. This can be done in -a variety of way. Here is one of them. +Stork system uses agents to monitor services. Stork Agent (`STAG` or simply `agent`) is a +daemon that is expected to be deployed and run on each machine to be monitored. As of Stork 0.2.0 +release there are no automated deployment routines and STAG has to be copied and run manually. +This can be done in a variety of ways. Here is one of them. Assuming you want to monitor services running on machine with IP 192.0.2.1, you can do the following on the Stork server command line: @@ -53,9 +53,9 @@ run it: ./stork-agent -You can optionally pass ``--host=`` or set the ``STORK_AGENT_ADDRESS`` variable to specify which -address the agent will listen on. You can pass ``--port`` or set the ``STORK_AGENT_PORT`` -variable to specify which TCP port the agent will listen on. +You can optionally pass ``--host=`` or set the ``STORK_AGENT_ADDRESS`` environment variable to +specify which address the agent will listen on. You can pass ``--port`` or set the ``STORK_AGENT_PORT`` +environment variable to specify which TCP port the agent will listen on. .. note:: @@ -65,7 +65,7 @@ variable to specify which TCP port the agent will listen on. Registering New Machine ======================= -Once the agent is deployed and running on the machine to be monitored, you should instruct stork +Once the agent is deployed and running on the machine to be monitored, you should instruct Stork server to start monitoring it. You can do so by going to Services menu and choosing Machines. You will be presented with a list of currently registered machines. @@ -90,7 +90,7 @@ To monitor registered machines, go to Services menu and click Machines. A list o registered machines will be displayed. Pagination mechanism is available to display larger number of machines. -There is a filtering mechanism that acts as an omnibox. The string typed is searched for in address, +There is a filtering mechanism that acts as an omnibox. The string typed is searched for an address, agent version, hostname, OS, platform, OS version, kernel version, kernel architecture, virtualization system, host-id fields. The filtering happens once you hit ENTER. -- GitLab