Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .github/workflows/mkdocs-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,13 @@ jobs:
mkdocs serve > /dev/null 2>&1 &
SERVER_PID=$!
echo "mk server in PID $SERVER_PID"
# Give enough time for deployment
sleep 30
# Give enough time for deployment (2min max)
for run in {1..120}; do
if curl --output /dev/null --silent --head --fail http://127.0.0.1:8000/; then
break
fi
sleep 1
done
echo "Launching linkchecker"
linkchecker --no-warnings --no-status http://127.0.0.1:8000/

Expand All @@ -81,4 +86,4 @@ jobs:
git config --global user.email mike@docs.hopsworks.ai

- name: Generate the docs with mike
run: mike deploy 3.2-SNAPSHOT dev -u
run: mike deploy test-version
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This is the source of the Hopsworks Documentation published at <https://docs.hop

## Build instructions

We use `mkdocs` together with [`mike`]((https://github.com/jimporter/mike/) for versioning to build the documentation.
We use `mkdocs` together with [`mike`](https://github.com/jimporter/mike/) for versioning to build the documentation.
We also use this two main mkdocs plugins: [`mkdocstrings`](https://mkdocstrings.github.io/) and [its Python handler](https://mkdocstrings.github.io/python/), and [`mkdocs-material`](https://squidfunk.github.io/mkdocs-material/) as the theme.

**Background about `mike`:**
Expand Down
17 changes: 10 additions & 7 deletions docs/setup_installation/admin/configure-project-mapping.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Configure group to project mapping

## Introduction

A group-to-project mapping lets you automatically add all members of a Hopsworks group to a project, eliminating the need to add each user individually. To create a mapping, you simply select a Hopsworks group, choose the project it should be linked to, and assign the role that its members will have within that project.

Once a mapping is created, project membership is controlled through Hopsworks group membership. Any updates made to the Hopsworks group—such as adding or removing users—will automatically be reflected in the project membership. For example, if a user is removed from the Hopsworks group, they will also be removed from the corresponding project.
Expand All @@ -9,6 +10,7 @@ Once a mapping is created, project membership is controlled through Hopsworks gr

1. Hopsworks group mapping sync enabled. This can be done by setting the variable ```hw_group_mapping_sync_enabled=true```.
See [Cluster Configuration](./variables.md) on how to change variable values in Hopsworks.

<figure>
<a href="../../../../assets/images/admin/project-mapping/configuration-variables.png">
<img src="../../../../assets/images/admin/project-mapping/configuration-variables.png" alt="Enable Hopsworks mapping" />
Expand All @@ -26,9 +28,10 @@ If you can not find the variable ```hw_group_mapping_sync_enabled``` create it b
</figure>

### Step 1: Create a mapping

To create a mapping go to **Cluster Settings** by clicking on your name in the top right
corner of the navigation bar and choosing *Cluster Settings* from the dropdown menu.
In the _Project mapping_ tab, you can create a new mapping by clicking on _Create new mapping_.
In the *Project mapping* tab, you can create a new mapping by clicking on *Create new mapping*.

<figure>
<a href="../../../../assets/images/admin/project-mapping/project-mapping-empty.png">
Expand All @@ -45,10 +48,10 @@ This will take you to the create mapping page shown below
<figcaption>Create mapping</figcaption>
</figure>

Here you can enter your Hopsworks group and map it to a project from the _Project_ drop down list.
You can also choose the _Project role_ users will be assigned when they are added to the project.
Here you can enter your Hopsworks group and map it to a project from the *Project* drop down list.
You can also choose the *Project role* users will be assigned when they are added to the project.

Finally, click on _Create mapping_ and go back to mappings. You should see the newly created mapping(s) as shown below.
Finally, click on *Create mapping* and go back to mappings. You should see the newly created mapping(s) as shown below.

<figure>
<a href="../../../../assets/images/admin/project-mapping/group-to-project-mappings.png">
Expand All @@ -59,7 +62,7 @@ Finally, click on _Create mapping_ and go back to mappings. You should see the n

### Step 2: Edit a mapping

From the list of mappings click on the edit button (:material-pencil:). This will open a popup that will allow you to change the _group_, _project name_, and _project role_ of a mapping.
From the list of mappings click on the edit button (:material-pencil:). This will open a popup that will allow you to change the *group*, *project name*, and *project role* of a mapping.

<figure>
<a href="../../../../assets/images/admin/project-mapping/edit-hw-mapping.png">
Expand All @@ -69,11 +72,11 @@ From the list of mappings click on the edit button (:material-pencil:). This wil
</figure>

!!!Warning
Updating a mapping's _group_ or _project name_ will remove all members of the previous group from the project.
Updating a mapping's *group* or *project name* will remove all members of the previous group from the project.

### Step 3: Delete a mapping

To delete a mapping click on the delete button.

!!!Warning
Deleting a mapping will remove all members of that group from the project.
Deleting a mapping will remove all members of that group from the project.
18 changes: 11 additions & 7 deletions docs/setup_installation/admin/oauth2/configure-project-mapping.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# Configure OAuth2 group to project mapping

## Introduction

A group-to-project mapping lets you automatically add all members of an OAuth2 group to a project, eliminating the need to add each user individually. To create a mapping, you simply select an OAuth2 group, choose the project it should be linked to, and assign the role that its members will have within that project.

Once a mapping is created, project membership is controlled through OAuth2 group membership. Any updates made to the OAuth2 group—such as adding or removing users—will automatically be reflected in Hopsworks. For example, if a user is removed from the OAuth2 group, they will also be removed from the corresponding project.

## Prerequisites

1. A server configured with OAuth2. See [Register Identity Provider in Hopsworks](./create-client.md) for instructions on how to do this.
2. OAuth2 group mapping sync enabled. This can be done by setting the variable ```oauth_group_mapping_sync_enabled=true```.
See [Cluster Configuration](../variables.md) on how to change variable values in Hopsworks.

<figure>
<a href="../../../../assets/images/admin/project-mapping/configuration-variables.png">
<img src="../../../../assets/images/admin/project-mapping/configuration-variables.png" alt="Enable OAuth2 mapping" />
Expand All @@ -26,9 +29,10 @@ If you can not find the variable ```oauth_group_mapping_sync_enabled``` create i
</figure>

### Step 1: Create a mapping

To create a mapping go to **Cluster Settings** by clicking on your name in the top right
corner of the navigation bar and choosing *Cluster Settings* from the dropdown menu.
In the _Project mapping_ tab, you can create a new mapping by clicking on _Create new mapping_.
In the *Project mapping* tab, you can create a new mapping by clicking on *Create new mapping*.

<figure>
<a href="../../../../assets/images/admin/project-mapping/project-mapping-empty.png">
Expand All @@ -45,10 +49,10 @@ This will take you to the create mapping page shown below
<figcaption>Create mapping</figcaption>
</figure>

Here you can enter your OAuth2 group and map it to a project from the _Project_ drop down list.
You can also choose the _Project role_ users will be assigned when they are added to the project.
Here you can enter your OAuth2 group and map it to a project from the *Project* drop down list.
You can also choose the *Project role* users will be assigned when they are added to the project.

Finally, click on _Create mapping_ and go back to mappings. You should see the newly created mapping(s) as shown below.
Finally, click on *Create mapping* and go back to mappings. You should see the newly created mapping(s) as shown below.

<figure>
<a href="../../../../assets/images/admin/project-mapping/group-to-project-mappings.png">
Expand All @@ -64,7 +68,7 @@ Finally, click on _Create mapping_ and go back to mappings. You should see the n

### Step 2: Edit a mapping

From the list of mappings click on the edit button (:material-pencil:). This will open a popup that will allow you to change the _remote group_, _project name_, and _project role_ of a mapping.
From the list of mappings click on the edit button (:material-pencil:). This will open a popup that will allow you to change the *remote group*, *project name*, and *project role* of a mapping.

<figure>
<a href="../../../../assets/images/admin/project-mapping/edit-mapping.png">
Expand All @@ -74,11 +78,11 @@ From the list of mappings click on the edit button (:material-pencil:). This wil
</figure>

!!!Warning
Updating a mapping's _remote group_ or _project name_ will remove all members of the previous group from the project.
Updating a mapping's *remote group* or *project name* will remove all members of the previous group from the project.

### Step 3: Delete a mapping

To delete a mapping click on the delete button.

!!!Warning
Deleting a mapping will remove all members of that group from the project.
Deleting a mapping will remove all members of that group from the project.
2 changes: 1 addition & 1 deletion docs/templates/python/material/attribute.html.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

{% block heading scoped %}
{% block source_link scoped %}
{% if config.link_source and attribute.source_link %}
{% if config.extra.link_source and attribute.source_link %}
<span style="float:right;font-size:.8rem;font-weight:400"><a href="{{ attribute.source_link }}">[source]</a></span>
{% endif %}
{% endblock source_link %}
Expand Down
2 changes: 1 addition & 1 deletion docs/templates/python/material/class.html.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

{% block heading scoped %}
{% block source_link scoped %}
{% if config.link_source and class.source_link %}
{% if config.extra.link_source and class.source_link %}
<span style="float:right;font-size:.8rem;font-weight:400"><a href="{{ class.source_link }}">[source]</a></span>
{% endif %}
{% endblock source_link %}
Expand Down
2 changes: 1 addition & 1 deletion docs/templates/python/material/function.html.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

{% block heading scoped %}
{% block source_link scoped %}
{% if config.link_source and function.source_link %}
{% if config.extra.link_source and function.source_link %}
<span style="float:right;font-size:.8rem;font-weight:400"><a href="{{ function.source_link }}">[source]</a></span>
{% endif %}
{% endblock source_link %}
Expand Down
2 changes: 1 addition & 1 deletion docs/templates/python/material/module.html.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

{% block heading scoped %}
{% block source_link scoped %}
{% if config.link_source and module.source_link %}
{% if config.extra.link_source and module.source_link %}
<span style="float:right;font-size:.8rem;font-weight:400"><a href="{{ module.source_link }}">[source]</a></span>
{% endif %}
{% endblock source_link %}
Expand Down
1 change: 0 additions & 1 deletion docs/user_guides/projects/jobs/pyspark_job.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,6 @@ The following table describes the job configuration parameters for a PYSPARK job
| <nobr>`conf['archives']`</nobr> | string | Comma-separated string of HDFS path(s) to archives to be made available to the application. Example: `hdfs:///Project/<project_name>/Resources/archive.zip,...` | `null` |
| <nobr>`conf['properties']`</nobr> | string | A new line separated (`\n`) list of properties to pass to the Spark application. The properties should be in the format `name=value` | `null` |


## Accessing project data

### Read directly from the filesystem (recommended)
Expand Down
1 change: 0 additions & 1 deletion docs/user_guides/projects/jobs/spark_job.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,6 @@ The following table describes the job configuration parameters for a SPARK job.
| <nobr>`conf['archives']`</nobr> | string | Comma-separated string of HDFS path(s) to archives to be made available to the application. Example: `hdfs:///Project/<project_name>/Resources/archive.zip,...` | `null` |
| <nobr>`conf['properties']`</nobr> | string | A new line separated (`\n`) list of properties to pass to the Spark application. The properties should be in the format `name=value` | `null` |


## Accessing project data

### Read directly from the filesystem (recommended)
Expand Down
3 changes: 2 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -346,9 +346,10 @@ plugins:
show_symbol_type_heading: true
show_symbol_type_toc: true
show_source: false
link_source: true
docstring_section_style: spacy
annotations_path: source
extra:
link_source: true
inventories:
- https://docs.python.org/3/objects.inv
- https://pandas.pydata.org/docs/objects.inv
Expand Down