The project leverages
docker-compose with a custom python script so you need to have the following packages installed in your machine:
docker - v1.13.0+
docker-compose - v1.23.2+
python - v3.6+
There are additional python dependencies (mentioned in the
requirements/pre-requirements.txtfile) that can be installed using the
In some systems you could find pre-installed older versions. Please check this and install a supported version before attempting the installation. Otherwise it would fail.
Obviously we strongly suggest reading through all the page to configure IntelOwl in the most appropriate way.
However, if you feel lazy, you could just install and test IntelOwl with the following steps. Be sure to run
python commands with
sudo if permissions/roles have not been set
Note: We’ve added a new script
initialize.sh that will check compatibility with your system and attempt to install the required dependencies.
# clone the IntelOwl project repository git clone https://github.com/intelowlproject/IntelOwl cd IntelOwl/ # construct environment files from templates cp docker/env_file_app_template docker/env_file_app cp docker/env_file_postgres_template docker/env_file_postgres cp docker/env_file_integrations_template docker/env_file_integrations cp frontend/public/env_template.js frontend/public/env.js # verify installed dependencies sudo ./initialize.sh # start the app sudo python3 start.py prod up # now the application is running on http://localhost:80 # create a super user sudo docker exec -ti intelowl_uwsgi python3 manage.py createsuperuser # now you can login with the created user form http://localhost:80 # Have fun!
WarningThe first time you start IntelOwl, a lot of database migrations are being applied. This requires some time. If you get 500 status code errors in the GUI, just wait few minutes and then refresh the page.
HintThere is a YouTube video that may help in the installation process. (ManySteps have changed since v2.0.0)
IntelOwl is composed of various different technologies, namely:
Rabbit-MQ: Message Broker
Celery: Task Queue
Nginx: Reverse proxy for the Django API and web asssets.
Uwsgi: Application Server
Elastic Search (optional): Auto-sync indexing of analysis’ results.
Kibana (optional): GUI for Elastic Search. We provide a saved configuration with dashboards and visualizations.
Flower (optional): Celery Management Web Interface
All these components are managed via docker-compose.
Open a terminal and execute below commands to construct new environment files from provided templates.
cp docker/env_file_app_template docker/env_file_app cp docker/env_file_postgres_template docker/env_file_postgres cp docker/env_file_integrations_template docker/env_file_integrations cp frontend/public/env_template.js frontend/public/env.js sudo ./initialize.sh
Environment configuration (required)
docker/env_file_app, configure different variables as explained below.
REQUIRED variables to run the image:
DB_PASSWORD: PostgreSQL configuration (The DB credentals should match the ones in the
env_file_postgres). If you like, you can configure the connection to an external PostgreSQL instance in the same variables. Then, to avoid to run PostgreSQL locally, please run IntelOwl with the option
postgresto have the app properly communicate with the PostgreSQL container.
Strongly recommended variable to set:
DJANGO_SECRET: random 50 chars key, must be unique. If you do not provide one, Intel Owl will automatically set a new secret on every run.
mywebsite.com): the web domain of your instance, this is used for generating links to analysis results.
OLD_JOBS_RETENTION_DAYS: Database retention for analysis results (default: 14 days). Change this if you want to keep your old analysis longer in the database.
Other optional configuration to enable specific services / features
Configuration required to enable integration with Slack:
SLACK_TOKEN: Slack token of your Slack application that will be used to send/receive notifications
DEFAULT_SLACK_CHANNEL: ID of the Slack channel you want to post the message to
Configuration required to enable Re-Captcha in the Login and the Registration Page:
USE_RECAPTCHA: if you want to use recaptcha on your login
RECAPTCHA_SECRET_KEY: your recaptcha secret key In the
RECAPTCHA_SITEKEY: Recaptcha Key for your site
Configuration required to have InteOwl sending Emails (registration requests, mail verification, password reset/change, etc)
DEFAULT_FROM_EMAIL: email address used for automated correspondence from the site manager (example:
DEFAULT_EMAIL: email address used for correspondence with users (example:
EMAIL_HOST: the host to use for sending email with SMTP
EMAIL_HOST_USER: username to use for the SMTP server defined in EMAIL_HOST
EMAIL_HOST_PASSWORD: password to use for the SMTP server defined in EMAIL_HOST. This setting is used in conjunction with EMAIL_HOST_USER when authenticating to the SMTP server.
EMAIL_PORT: port to use for the SMTP server defined in EMAIL_HOST.
EMAIL_USE_TLS: whether to use an explicit TLS (secure) connection when talking to the SMTP server, generally used on port 587.
EMAIL_USE_SSL: whether to use an implicit TLS (secure) connection when talking to the SMTP server, generally used on port 465.
Database configuration (required if running PostgreSQL locally)
If you use a local PostgreSQL instance (this is the default), in the
env_file_postgres you have to configure different variables as explained below.
Logrotate configuration (strongly recommended)
If you want to have your logs rotated correctly, we suggest you to add the configuration for the system Logrotate.
To do that you can leverage the
initialize.sh script. Otherwise, if you have skipped that part, you can manually install logrotate by launching the following script:
cd ./docker/scripts ./install_logrotate.sh
We decided to do not leverage Django Rotation Configuration because it caused problematic concurrency issues, leading to logs that are not rotated correctly and to apps that do not log anymore. Logrotate configuration is more stable.
Crontab configuration (recommended for advanced deployments)
We added few Crontab configurations that could be installed in the host machine at system level to solve some possible edge-case issues:
Memory leaks: Once a week it is suggested to do a full restart of the application to clean-up the memory used by the application. Practical experience suggest us to do that to solve some recurrent memory issues in Celery. This cron (
application_restart) assumes that you have executed IntelOwl with the parameters
-all_analyzers. If you didn’t, feel free to change the cron as you wish.
This configuration is optional but strongly recommended for people who want to have a production grade deployment. To install it you need to run the following script in each deployed server:
cd ./docker/scripts ./install_crontab.sh
Web server configuration (required for enabling HTTPS)
Intel Owl provides basic configuration for:
In case you enable HTTPS, remember to set the environment variable
HTTPS_ENABLED as “enabled” to increment the security of the application.
There are 3 options to execute the web server:
HTTP only (default)
The project would use the default deployment configuration and HTTP only.
HTTPS with your own certificate
The project provides a template file to configure Nginx to serve HTTPS:
You should change
server_namein that file and put those required files in the specified locations.
Then you should call the
start.pyscript with the parameter
--httpsto leverage the right Docker Compose file for HTTPS.
Plus, if you use Flower, you should change in the
HTTPS with Let’s Encrypt
We provide a specific docker-compose file that leverages Traefik to allow fast deployments of public-faced and HTTPS-enabled applications.
Before using it, you should configure the configuration file
docker/traefik.override.ymlby changing the email address and the hostname where the application is served. For a detailed explanation follow the official documentation: Traefix doc.
After the configuration is done, you can add the option
--traefikwhile executing the
Important InfoIntelOwl depends heavily on docker and docker compose so as to hide this complexity from the enduser the project leverages a custom script (
start.py) to interface with
You may invoke
$ python3 start.py –help to get help and usage info.
The CLI provides the primitives to correctly build, run or stop the containers for IntelOwl. Therefore,
Now that you have completed different configurations, starting the containers is as simple as invoking:
$ python3 start.py prod up
You can add the parameter
-d to run the application in the background.
HintStarting from IntelOwl 4.0.0, with the startup script you can select which version of IntelOwl you want to run (
--version). This can be helpful to keep using old versions in case of retrocompatibility issues. The
--versionparameter checks out the Git Repository to the Tag of the version that you have chosen. This means that if you checkout to a v3.x.x version, you won't have the
--versionparameter anymore so you would need to manually checkout back to the
masterbranch to use newer versions.
To stop the application you have to:
if executed without
if executed with
python3 start.py prod down
Cleanup of database and application
This is a destructive operation but can be useful to start again the project from scratch
python3 start.py prod down -v
You may want to run
docker exec -ti intelowl_uwsgi python3 manage.py createsuperuser after first run to create a superuser.
Then you can add other users directly from the Django Admin Interface after having logged with the superuser account.
To manage users, organizations and their visibility please refer to this section
Update and Rebuild
Rebuilding the project / Creating custom docker build
If you make some code changes and you like to rebuild the project, follow these steps:
python3 start.py test build --tag=<your_tag> .to build the new docker image.
Add this new image tag in the
Start the containers with
python3 start.py test up --build.
Update to the most recent version
To update the project with the most recent available code you have to follow these steps:
$ cd <your_intel_owl_directory> # go into the project directory $ git pull # pull new changes $ python3 start.py prod down # kill and destroy the currently running IntelOwl containers $ python3 start.py prod up # restart the IntelOwl application
NoteAfter an upgrade, sometimes a database error in Celery Containers could happen. That could be related to new DB migrations which are not applied by the main Uwsgi Container yet. Do not worry. Wait few seconds for the Uwsgi container to start correctly, then put down the application again and restart it. The problem should be solved. If not, please feel free to open an issue on Github
NoteAfter having upgraded IntelOwl, in case the application does not start and you get an error like this:
PermissionError: [Errno 13] Permission denied: '/var/log/intel_owl/django/authentication.log
just run this:
sudo chown -R www-data:www-data /var/lib/docker/volumes/intel_owl_generic_logs/_data/django
and restart IntelOwl. It should solve the permissions problem.
WarningMajor versions of IntelOwl are usually incompatible from one another. Maintainers strive to keep the upgrade between major version easy but it's not always like that. Below you can find the additional process required to upgrade from each major versions.
Updating to >=5.0.0 from a 4.x.x version
IntelOwl v5 introduced some major changes regarding how the plugins and their related configuration are managed in the application. Before upgrading, some important things should be checked by the administrator:
A lot of database migrations will need to be applied. Just be patient few minutes once you install the new major release. If you get 500 status code errors in the GUI, just wait few minutes and then refresh the page.
We moved away from the old big
analyzer_config.jsonwhich was storing all the base configuration of the Analyzers to a database model (we did the same for all the other plugins types too). This allows us to manage plugins creation/modification/deletion in a more reliable manner and via the Django Admin Interface. If you have created custom plugins and changed those
<plugins>_config.jsonfile manually, you would need to re-create those custom plugins again from the Django Admin Interface. To do that please follow the related new documentation
We have REMOVED all the analyzers that we deprecated during the v4 releases cycle. Please substitute them with their respective new names, in case they have a replacement.
Pulsedive_Active_IOCanalyzer. Please substitute it with the new
Fortiguardanalyzer because endpoint does not work anymore. No substitute.
Rendertronanalyzer not working as intended. No substitute.
Robtexvarious analyzers and substituted with new versions.
Doc_Info_Experimental. Its functionality (XLM Macro parsing) is moved to
Strings_Info_Classic. Please use
Strings_Info_ML. Please use
Strings_Infoand set the parameter
Yara_Scan_<repo>analyzers. They all went merged in the single
Darksearch_Queryanalyzer because the service does not exist anymore. No substitute.
UnpacMe_EXE_Unpacker. Please use
Anomali_Threatstream_<option>analyzers. Now we have a single
Anomali_Threatstreamanalyzer. Use the parameters to select the specific API you need.
Updating to >=5.0.0 from a 3.x.x version
This is not supported. Please perform a major upgrade once at a time.
Updating to >=4.0.0 from a 3.x.x version
IntelOwl v4 introduced some major changes regarding the permission management, allowing an easier way to manage users and visibility. But that did break the previous available DB. So, to migrate to the new major version you would need to delete your DB. To do that, you would need to delete your volumes and start the application from scratch.
python3 start.py prod down -v
Please be aware that, while this can be an important effort to manage, the v4 IntelOwl provides an easier way to add, invite and manage users from the application itself. See the Organization section.
Updating to >=2.0.0 from a 1.x.x version
Users upgrading from previous versions need to manually move
env_file_integrations files under the new
Updating to >v1.3.x from any prior version
If you are updating to >v1.3.0 from any prior version, you need to execute a helper script so that the old data present in the database doesn’t break.
Follow the above updation steps, once the docker containers are up and running execute the following in a new terminal
docker exec -ti intelowl_uwsgi bash
to get a shell session inside the IntelOwl’s container.
Now just copy and paste the below command into this new session,
curl https://gist.githubusercontent.com/Eshaan7/b111f887fa8b860c762aa38d99ec5482/raw/758517acf87f9b45bd22f06aee57251b1f3b1bbf/update_to_v1.3.0.py | python -
If you see “Update successful!”, everything went fine and now you can enjoy the new features!