Advanced Configuration

This page includes details about some advanced features that Intel Owl provides which can be optionally configured by the administrator.

• ElasticSearch

  • Kibana

  • Example Configuration

• Authentication options

  • OAuth support

  • LDAP

  • RADIUS

• OpenCTI support

• Cloud Support

  • AWS support

    • Secrets

    • SQS

    • S3

    • SES

  • Google Kubernetes Engine

• Queues

  • Multi Queue

  • Queue Customization

  • Queue monitoring

ElasticSearch

Intel Owl makes use of django-elasticsearch-dsl to index Job results into elasticsearch. The save and delete operations are auto-synced so you always have the latest data in ES.

In the env_file_app_template, you’d see various elasticsearch related environment variables. The user should spin their own Elastic Search instance and configure these variables.

Kibana

Intel Owl provides a Kibana’s “Saved Object” configuration (with example dashboard and visualizations). It can be downloaded from here and can be imported into Kibana by going to the “Saved Objects” panel (http://localhost:5601/app/management/kibana/objects).

Example Configuration

1. Setup Elastic Search and Kibana and say it is running in a docker service with name elasticsearch on port 9200 which is exposed to the shared docker network. (Alternatively, you can spin up a local Elastic Search instance, by appending --elastic to the python3 start.py ... command. Note that the local Elastic Search instance consumes large amount of memory, and hence having >=16GB is recommended.))

2. In the env_file_app, we set ELASTICSEARCH_ENABLED to True and ELASTICSEARCH_HOST to elasticsearch:9200.

3. Configure the version of the ElasticSearch Library used depending on the version of our Elasticsearch server. This is required for compatibility. To do that, you can leverage the option --pyelastic-version of the start.py script. The default value of that parameter indicates the version that would be installed by default.

4. Rebuild the docker images with python3 start.py test --pyelastic-version x.x.x build (required only if you changed the default value of --pyelastic-version)

5. Now start the docker containers and execute,

docker exec -ti intelowl_uwsgi python manage.py search_index --rebuild

This will build and populate all existing job objects into the jobs index.

Authentication options

IntelOwl provides support for some of the most common authentication methods:

• Google Oauth2

• LDAP

• RADIUS

Google OAuth2

The first step is to create a Google Cloud Platform project, and then create OAuth credentials for it.

It is important to add the correct callback in the “Authorized redirect URIs” section to allow the application to redirect properly after the successful login. Add this:

http://<localhost|yourowndomain>/api/auth/google-callback

After that, specify the client ID and secret as GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET environment variables and restart IntelOwl to see the applied changes.

Note

While configuring Google Auth2 you can choose either to enable access to the all users with a Google Account ("External" mode) or to enable access to only the users of your organization ("Internal" mode). Reference

LDAP

IntelOwl leverages Django-auth-ldap to perform authentication via LDAP.

How to configure and enable LDAP on Intel Owl?

1. Change the values with your LDAP configuration inside configuration/ldap_config.py. This file is mounted as a docker volume, so you won’t need to rebuild the image.

Note

For more details on how to configure this file, check the official documentation of the django-auth-ldap library.

1. Once you have done that, you have to set the environment variable LDAP_ENABLED as True in the environment configuration file env_file_app. Finally, you can restart the application with docker-compose up

RADIUS Authentication

IntelOwl leverages Django-radius to perform authentication via RADIUS server.

How to configure and enable RADIUS authentication on Intel Owl?

1. Change the values with your RADIUS auth configuration inside configuration/radius_config.py. This file is mounted as a docker volume, so you won’t need to rebuild the image.

Note

For more details on how to configure this file, check the official documentation of the django-radius library.

1. Once you have done that, you have to set the environment variable RADIUS_AUTH_ENABLED as True in the environment configuration file env_file_app. Finally, you can restart the application with docker-compose up

OpenCTI

Like many other integrations that we have, we have an Analyzer and a Connector for the OpenCTI platform.

This allows the users to leverage these 2 popular open source projects and frameworks together.

So why we have a section here? This is because there are various compatibility problems with the official PyCTI library.

We found out (see issues in IntelOwl and PyCTI) that, most of the times, it is required that the OpenCTI version of the server you are using and the pycti version installed in IntelOwl must match perfectly.

Because of that, we decided to provide to the users the chance to customize the version of PyCTI installed in IntelOwl based on the OpenCTI version that they are using.

To do that, you would need to leverage the option --pycti-version provided by the script.py helper:

• check the default version that would be installed by checking the description of the option --pycti-version with python3 start.py -h

• if the default version is different from your OpenCTI server version, you need to rebuild the IntelOwl Image with python3 start.py test --pycti-version <your_version> build

• then restart the project python3 start.py test up --build

• enjoy

Cloud Support

AWS support

We have support for several AWS services.

You can customize the AWS Region location of you services by changing the environment variable AWS_REGION. Default is eu-central-1

You have to add some credentials for AWS: if you have IntelOwl deployed on the AWS infrastructure, you can use IAM credentials: to allow that just set AWS_IAM_ACCESS to True. If that is not the case, you have to set both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

S3

If you prefer to use S3 to store the analyzed samples, instead of the local storage, you can do it.

First, you need to configure the environment variable LOCAL_STORAGE to False to enable it and set AWS_STORAGE_BUCKET_NAME to the AWS bucket you want to use.

Then you need to configure permission access to the chosen S3 bucket.

SQS

If you like, you could use AWS SQS instead of Rabbit-MQ to manage your queues. In that case, you should create new SQS queues in AWS called intelowl-<environment>-<queue_name> and give your instances on AWS the proper permissions to access it. Only FIFO queues are supported.

Also, you need to set the environment variable AWS_SQS to True and populate the AWS_USER_NUMBER. This is required to connect in the right way to the selected SQS queues.

Ultimately, to avoid to run RabbitMQ locally, you would need to use the option --use-external-broker when launching IntelOwl with the start.py script.

RDS

If you like, you could use AWS RDS instead of PostgreSQL for your database. In that case, you should change the database required options accordingly: DB_HOST, DB_PORT, DB_USER, DB_PASSWORD and setup your machine to access the service.

If you have IntelOwl deployed on the AWS infrastructure, you can use IAM credentials to access the Postgres DB. To allow that just set AWS_RDS_IAM_ROLE to True. In this case DB_PASSWORD is not required anymore.

Moreover, to avoid to run PostgreSQL locally, you would need to use the option --use-external-database when launching IntelOwl with the start.py script.

SES

If you like, you could use Amazon SES for sending automated emails (password resets / registration requests, etc).

You need to configure the environment variable AWS_SES to True to enable it.

Secrets

You can use the “Secrets Manager” to store your credentials. In this way your secrets would be better protected.

Instead of adding the variables to the environment file, you should just add them with the same name on the AWS Secrets Manager and Intel Owl will fetch them transparently.

Obviously, you should have created and managed the permissions in AWS in advance and accordingly to your infrastructure requirements.

Also, you need to set the environment variable AWS_SECRETS to True to enable this mode.

NFS

You can use a Network File System for the shared_files that are downloaded runtime by IntelOwl (for example Yara rules).

To use this feature, you would need to add the address of the remote file system inside the .env file, and you would need to use the option --nfs when launching IntelOwl with the start.py script.

Google Kubernetes Engine

Right now there is no official support for Kubernetes deployments.

But we have an active community. Please refer to the following blog post for an example on how to deploy IntelOwl on Google Kubernetes Engine:

Deploying Intel-Owl on GKE by Mayank Malik.

Queues

Multi Queue

IntelOwl provides an additional multi-queue.override.yml compose file allowing IntelOwl users to better scale with the performance of their own architecture.

If you want to leverage it, you should add the option --multi-queue when starting the project. Example:

python3 start.py prod --multi-queue up

This functionality is not enabled by default because this deployment would start 2 more containers so the resource consumption is higher. We suggest to use this option only when leveraging IntelOwl massively.

Queue Customization

It is possible to define new celery workers: each requires the addition of a new container in the docker-compose file, as shown in the multi-queue.override.yml.

Moreover IntelOwl requires that the name of the workers are provided in the docker-compose file. This is done through the environment variable CELERY_QUEUES inside the uwsgi container. Each queue must be separated using the character ,, as shown in the example.

One can customize what analyzer should use what queue by specifying so in the analyzer entry in the analyzer_config.json configuration file. If no queue(s) are provided, the default queue will be selected.

Queue monitoring

IntelOwl provides an additional flower.override.yml compose file allowing IntelOwl users to use Flower features to monitor and manage queues and tasks

If you want to leverage it, you should add the option --flower when starting the project. Example:

python3 start.py prod --flower up

The flower interface is available at port 5555: to set the credentials for its access, update the environment variables

FLOWER_USER
FLOWER_PWD

or change the .htpasswd file that is created in the docker directory in the intelowl_flower container.