Advanced Usage

This page includes details about some advanced features that Intel Owl provides which can be optionally enabled. Namely,

Optional Analyzers

Some analyzers which run in their own Docker containers are kept disabled by default. They are disabled by default to prevent accidentally starting too many containers and making your computer unresponsive.

Name Analyzers Description
Malware Tools Analyzers
  • PEframe_Scan
  • Capa_Info
  • Floss
  • Strings_Info_Classic, Strings_Info_ML
  • Manalyze
  • ClamAV
  • Thug_URL_Info, Thug_HTML_Info
  • BoxJS_Scan_JavaScript
  • APKiD_Scan_APK_DEX_JAR
  • Qiling_Windows, Qiling_Windows_Shellcode, Qiling_Linux, Qiling_Linux_Shellcode
  • PEFrame performs static analysis on Portable Executable malware and malicious MS Office documents
  • Capa detects capabilities in executable files
  • FLOSS automatically deobfuscate strings from malware binaries
  • String_Info_Classic extracts human-readable strings where as ML version of it ranks them
  • Manalyze statically analyzes PE (Portable-Executable) files in-depth
  • ClamAV antivirus engine scans files for trojans, viruses, malwares using a multi-threaded daemon
  • Thug performs hybrid dynamic/static analysis on a URL or HTML page.
  • Box-JS is a tool for studying JavaScript malware
  • APKiD identifies many compilers, packers, obfuscators, and other weird stuff from an APK or DEX file
  • Qiling is a tool for emulating the execution of a binary file or a shellcode. It requires the configuration of its rootfs, and the optional configuration of profiles. The rootfs can be copied from the Qiling project: please remember that Windows dll must be manually added for license reasons. Qiling provides a DllCollector to retrieve dlls from your licensed Windows. Profiles must be placed in the profiles subfolder
TOR Analyzers Onionscan Scans TOR .onion domains for privacy leaks and information disclosures.
Renderton Renderton get screenshot of a web page using rendertron (a headless chrome solution using puppeteer). Configuration variables have to be included in the `config.json`, see config options of renderton . To use a proxy, include an argument --proxy-server=YOUR_PROXY_SERVER in puppeteerArgs.
CyberChef CyberChef Run a transformation on a CyberChef server using pre-defined or custom recipes(rules that describe how the input has to be transformed). Check further instructions here
PCAP Analyzers Suricata You can upload a PCAP to have it analyzed by Suricata with the open Ruleset. The result will provide a list of the triggered signatures plus a more detailed report with all the raw data generated by Suricata. You can also add your own rules (See paragraph "Analyzers with special configuration"). The installation is optimized for scaling so the execution time is really fast.

To enable all the optional analyzers you can add the option --all_analyzers when starting the project. Example:

python3 start.py prod --all_analyzers up

Otherwise you can enable just one of the cited integration by using the related option. Example:

python3 start.py prod --tor_analyzers up

Customize analyzer execution

Some analyzers and connectors provide the chance to customize the performed analysis based on parameters (params attr in the configuration file) that are different for each analyzer.

  • You can set a custom default values by changing their value attribute directly from the configuration files. Since IntelOwl v4, it is possible to change these values directly from the GUI in the section “Your plugin configuration”.

  • You can choose to provide runtime configuration when requesting an analysis that will be merged with the default overriding it. This override is done only for the specific analysis.

Info

Connectors parameters can only be changed from either their configuration file or the "Your plugin configuration" section, not at the time of analysis request.

View and understand different parameters

To see the list of these parameters:

  • You can view the “Plugin” Section in IntelOwl to have a complete and updated view of all the options available

  • You can view the raw JSON configuration file, here.

from the GUI

You can click on “CUSTOMIZE ANALYZERS PARAMETERS” button and add the runtime configuration in the form of a dictionary. Example:

"VirusTotal_v3_Get_File": {
    "force_active_scan_if_old": true
}

from Pyintelowl

While using send_observable_analysis_request or send_file_analysis_request endpoints, you can pass the parameter runtime_configuration with the optional values. Example:

runtime_configuration = {
    "Doc_Info": {
        "additional_passwords_to_check": ["passwd", "2020"]
    }
}
pyintelowl_client.send_file_analysis_request(..., runtime_configuration=runtime_configuration)

CyberChef

You can either use pre-defined recipes or create your own as explained here.

To use a pre-defined recipe, set the predefined_recipe_name argument to the name of the recipe as defined here. Else, leave the predefined_recipe_name argument empty and set the custom_recipe argument to the contents of the recipe you want to use.

Additionally, you can also (optionally) set the output_type argument.

Pre-defined recipes

  • “to decimal”: [{"op": "To Decimal", "args": ["Space", False]}]

Analyzers with special configuration

Some analyzers could require a special configuration:

  • GoogleWebRisk: this analyzer needs a service account key with the Google Cloud credentials to work properly. You should follow the official guide for creating the key. Then you can copy the generated JSON key file in the directory configuration of the project and change its name to service_account_keyfile.json. This is the default configuration. If you want to customize the name or the location of the file, you can change the environment variable GOOGLE_APPLICATION_CREDENTIALS in the env_file_app file.

  • ClamAV: this Docker-based analyzer using clamd daemon as it’s scanner, communicating with clamdscan utility to scan files. The daemon requires 2 different configuration files: clamd.conf(daemon’s config) and freshclam.conf (virus database updater’s config). These files are mounted as docker volumes in /integrations/malware_tools_analyzers/clamav and hence, can be edited by the user as per needs, without restarting the application.

  • Suricata: you can customize the behavior of Suricata:

    • /integrations/pcap_analyzers/config/suricata/rules: here there are Suricata rules. You can change the custom.rules files to add your own rules at any time. Once you made this change, you need to either restart IntelOwl or (this is faster) run a new analysis with the Suricata analyzer and set the parameter reload_rules to true.

    • /integrations/pcap_analyzers/config/suricata/etc: here there are Suricata configuration files. Change it based on your wish. Restart IntelOwl to see the changes applied.

  • Yara_Scan_Custom_Signatures: you can use this pre-defined analyzer to run your own YARA signatures, either custom or imported. Just upload the .yar files with the signatures in the directory /configuration/custom_yara. That directory is mounted as a bind volume in Docker so you do not need to do anything to see the changes in the application.

Organizations and data sharing

Organizations are a great way to share data and analysis only with the members of your team. Invite the people you work with in your organization!

By default, analysis (jobs) are executed with a level of TLP that is WHITE. This means that these jobs are public and every IntelOwl user can see them. Thanks to the “Organization” feature, you can restrict the people who can see the analysis that you made.

How you can do that? Jobs with either AMBER or RED TLP value will be accessible to only members within the same organization. You can select the TLP for the analysis at the time of request.

Notifications

Since v4, IntelOwl integrated the notification system from the certego_saas package, allowing the admins to create notification that every user will be able to see.

The user would find the Notifications button on the top right of the page:

There the user can read notifications provided by either the administrators or the IntelOwl Maintainers.

As an Admin, if you want to add a notification to have it sent to all the users, you have to login to the Django Admin interface, go to the “Notifications” section and add it there. While adding a new notification, in the body section it is possible to even use HTML syntax, allowing to embed images, links, etc; in the app_name field, please remember to use intelowl as the app name.

Everytime a new release is installed, once the backend goes up it will automatically create a new notification, having as content the latest changes described in the CHANGELOG.md, allowing the users to keep track of the changes inside intelowl itself.

Authentication options

IntelOwl provides support for some of the most common authentication methods:

  • Google Oauth2

  • LDAP

  • RADIUS

Google OAuth2

The first step is to create a Google Cloud Platform project, and then create OAuth credentials for it.

After that, specify the client ID and secret as GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET environment variables.

Note

While configuring Google Auth2 you can choose either to enable access to the all users with a Google Account ("External" mode) or to enable access to only the users of your organization ("Internal" mode). Reference

LDAP

IntelOwl leverages Django-auth-ldap to perform authentication via LDAP.

How to configure and enable LDAP on Intel Owl?

  1. Change the values with your LDAP configuration inside configuration/ldap_config.py. This file is mounted as a docker volume, so you won’t need to rebuild the image.

Note

For more details on how to configure this file, check the official documentation of the django-auth-ldap library.
  1. Once you have done that, you have to set the environment variable LDAP_ENABLED as True in the environment configuration file env_file_app. Finally, you can restart the application with docker-compose up

RADIUS Authentication

IntelOwl leverages Django-radius to perform authentication via RADIUS server.

How to configure and enable RADIUS authentication on Intel Owl?

  1. Change the values with your RADIUS auth configuration inside configuration/radius_config.py. This file is mounted as a docker volume, so you won’t need to rebuild the image.

Note

For more details on how to configure this file, check the official documentation of the django-radius library.
  1. Once you have done that, you have to set the environment variable RADIUS_AUTH_ENABLED as True in the environment configuration file env_file_app. Finally, you can restart the application with docker-compose up

Google Kubernetes Engine deployment

Refer to the following blog post for an example on how to deploy IntelOwl on Google Kubernetes Engine:

Deploying Intel-Owl on GKE by Mayank Malik.

Queues

Multi Queue

IntelOwl provides an additional multi-queue.override.yml compose file allowing IntelOwl users to better scale with the performance of their own architecture.

If you want to leverage it, you should add the option --multi-queue when starting the project. Example:

python3 start.py prod --multi-queue up

This functionality is not enabled by default because this deployment would start 2 more containers so the resource consumption is higher. We suggest to use this option only when leveraging IntelOwl massively.

Queue Customization

It is possible to define new celery workers: each requires the addition of a new container in the docker-compose file, as shown in the multi-queue.override.yml.

Moreover IntelOwl requires that the name of the workers are provided in the docker-compose file. This is done through the environment variable CELERY_QUEUES inside the uwsgi container. Each queue must be separated using the character ,, as shown in the example.

One can customize what analyzer should use what queue by specifying so in the analyzer entry in the analyzer_config.json configuration file. If no queue(s) are provided, the default queue will be selected.

Queue monitoring

IntelOwl provides an additional flower.override.yml compose file allowing IntelOwl users to use Flower features to monitor and manage queues and tasks

If you want to leverage it, you should add the option --flower when starting the project. Example:

python3 start.py prod --flower up

The flower interface is available at port 5555: to set the credentials for its access, update the environment variables

FLOWER_USER
FLOWER_PWD

or change the .htpasswd file that is created in the docker directory in the intelowl_flower container.

AWS support

At the moment there’s a basic support for some of the AWS services. More is coming in the future.

Secrets

If you would like to run this project on AWS, I’d suggest you to use the “Secrets Manager” to store your credentials. In this way your secrets would be better protected.

This project supports this kind of configuration. Instead of adding the variables to the environment file, you should just add them with the same name on the AWS Secrets Manager and Intel Owl will fetch them transparently.

Obviously, you should have created and managed the permissions in AWS in advance and accordingly to your infrastructure requirements.

Also, you need to set the environment variable AWS_SECRETS to True to enable this mode.

You can customize the AWS Region changing the environment variable AWS_REGION.

SQS

If you like, you could use AWS SQS instead of Rabbit-MQ to manage your queues. In that case, you should change the parameter BROKER_URL to sqs:// and give your instances on AWS the proper permissions to access it.

Also, you need to set the environment variable AWS_SQS to True to activate the additional required settings.

S3

If you prefer to use S3 to store the samples, instead of a local storage, you can now do it.

First, you need to configure the environment variable LOCAL_STORAGE to False to enable it and set AWS_STORAGE_BUCKET_NAME to the proper AWS bucket. Then you have to add some credentials for AWS: if you have IntelOwl deployed on the AWS infrastructure, you can use IAM credentials: to allow that just set AWS_IAM_ACCESS to True. If that is not the case, you have to set both AWS_ACESS_KEY_ID and AWS_SECRET_ACCESS_KEY