Skip to main content

How can we help you?

Search

help Knowledge Base
forum Community
checklist Release Notes
event Events

Data Export configuration

Minetta Gould
  • Updated

    In this article

The Data Export feature enables Organizations and MSPs to export data at regularly timed intervals and configure exports to Amazon S3 or Splunk. After configuration, the data is available in the CSV version of the Query Log multiple times an hour. To export CyberSight endpoint activity data, see CyberSight Data Export configuration. 

Applies to accounts with the Data Export add-on enabled. Data Export is not included as a feature through distributors. For access to this feature, sign up with DNSFilter directly.

DNSFilter account users with admin permissions or higher configure Data Export.

Example Standard and Roaming Client exports and the data names are available to help navigate Data Export's capabilities. 

Data Export also combines Query Log data with other data for monitoring, action, and alerting, and integrates directly with most SIEMs, including:

Data Export can connect to S3 compatible services such as Wasabi and Backblaze.

This video was produced in 2022. The look and feel of our app has changed, but the function of the tool remains the same.

Enable Data Export

MSP Data Exports

Sales-Managed MSP accounts can purchase and enable Data Export for specific Organizations. Contact the Sales Team for details.

Self-Service MSP accounts cannot enable Data Export for individual Organizations. When activated, it applies to all Organizations included in the corresponding plan tier, such as all purchased Enterprise licenses across Organizations. Admins can still export data for a specific Organization, which supports sharing logs or reports with individual customers even though the feature is enabled globally.

The Upgrade Your Plan screen on the Data Export page indicates the feature has not yet been activated for the account.

To enable Data Export:

  1. From the DNSFilter dashboard, navigate to Organization (MSPs select MSP) and select Billing.
  2. Select Activated to activate Data Export.
  3. Select Save.

Amazon S3 Data Export configuration

For Amazon S3 setups, the call to upload files to S3 triggers multiple times an hour depending on the amount of data and system scaling. Each file is a maximum of 10k records.

The Data Export IP addresses provide the End Point destination for this setup. This is an optional field for some third-party resellers.

  1. From the DNSFilter dashboard, navigate to Tools and select Data Export.
  2. Select Configure Data Export.
  3. Select the Amazon S3 service.
  4. Select Continue.
  5. Enter the Amazon S3 Bucket name.
    This is a unique name for the organization's account that hosts information. See Amazon's guide for where to locate it.
  6. Enter optional parameters if applicable:
    1. Key Prefix value, used to organize the data stored in Amazon S3 buckets
    2. Endpoint fully qualified URL
  7. Enter the S3 Bucket Region (for example, us-east-1)
  8. Add authentication credentials, either an IAM role ARN or an Access Key ID and Secret Access Key:
    •  
      1. From the AWS Console, navigate to the IAM section and select Policies.
      2. Select a new policy

        1. Select the JSON editor and enter the policy below, replacing CUSTOMER_BUCKET_NAME_HERE with the name of the export bucket.

          {
    "Statement": [
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Effect": "Allow",
            "Resource": 
"arn:aws:s3:::CUSTOMER_BUCKET_NAME_HERE/*"
        }
    ],
    "Version": "2012-10-17"
}
        2. Give the policy a meaningful name and note it for referencing in the next steps.
      3. Create a new role.

        1. Under Select trusted entity, enter this JSON text, updating ORG_ID_HERE to the organization ID.

          { 
    "Version": "2012-10-17", 
    "Statement": [ 
        { 
            "Effect": "Allow", 
            "Principal": { 
                "AWS": "arn:aws:iam::968519194283:role/dnsfilter-external-access-role" 
            }, 
            "Action": "sts:AssumeRole", 
            "Condition": { 
                "StringEquals": { 
                    "sts:ExternalId": "ORG_ID_HERE" 
                } 
            } 
        } 
    ] 
}
        2. Select Next to Add permissions.
      4. Select the policy created above.
      5. Name the role and give it a description and applicable tags.
      6. Copy the role ARN from the policy view and add it to the DNSFilter dashboard configuration.
    • Enter the Access Key ID and Secret Access Key values: refer to the AWS security credentials page for more details on how to generate an Access Key.
  9. Select Verify & Test Account to test the connection.
  10. Select Finalize to complete the process.

Data can now export to S3. If an error message displays during setup, see the troubleshooting guide.

Zadara Data Export configuration

Configure data exports with Zadara following setup steps similar to S3.

To configure Data Export for Zadara:

  1. From Zadara Settings, enable Containers Virtual-Hosted Style Support.
  2. Navigate to User Information and collect the Endpoint (Public API Endpoint), Access, and Secret Key information used to connect DNSFilter and Zadara.

  3. Navigate to DNSFilter's Data Export page to complete the process. 

    Add https:// to the endpoint value to avoid connection errors. 

Splunk Data Export configuration

Splunk's HTTP Event Collector (HEC) API is utilized, which uses a well-recognized protocol for transferring data. It is scalable, secure, token-based for convenience, and easy to maintain.

The protocol is often implemented by SIEMs and data tools apart from Splunk, and may work out of the box with many preferred data tools.

For example, Humio implements a one-to-one HEC API which is already confirmed to work with this Data Export feature.

You will need the Data Export IP addresses to provide the End Point destination for this setup. 

To configure Data Export for Splunk:

  1. From the DNSFilter dashboard, navigate to Tools and select Data Export.
  2. Select Configure Data Export.
  3. Select the Splunk service.
  4. Select Continue.
  5. If applicable, turn off Use compressed data (default is enabled.

  6. Enter the Splunk account HTTP Event Collector URL and Active Event Collector Token. 

    See Splunk's Getting Data In documentation for information on how to generate an HTTP Event Collector URL and Token.

  7. Select Verify & Test Account to test the connection.
  8. Select Finalize to complete the process.

Data can now export to Splunk. If an error message displays during setup, see the troubleshooting guide. 

Restart a paused Data Export

A Data Export moves to a Paused state when the destination becomes unreachable (for example, after the SIEM goes offline during maintenance). Restart it as soon as the SIEM is reachable again. Two options are available:

Option 1: Restart from the dashboard

To restart a paused export from the dashboard:

  1. From the DNSFilter dashboard, navigate to Tools and select Data Export.

  2. Select Edit on the paused export.

  3. Verify the connection settings are correct.

  4. Select Verify & Test Account.

  5. Select Finalize.

Option 2: Restart with the API

The API call clears the export's errors and restarts it with the saved connection settings. SIEM credentials do not need to be re-entered. A successful call returns 200 with an empty body. Replace {organization_id} with the ID of the organization whose export is paused.

Note: The API key must belong to a user with the Owner or Administrator role.

Comments

4 comments

Date Votes
  • ARXSystems.io Harrington

    Why can't this be enabled through distributors? we were told both from pax8 and from dnsfilter that no functionality would be lost, but we need this enabled now and find out we can't turn it on? 

    0
  • Chris Todd Community Manager DNSFilter Staff
    Community Regular

    Hi ARXSystems.io Harrington -

    That's definitely frustrating to get conflicting information! I'm glad to see you're already in touch with our support team. They're the best resource to move this forward and get you setup corretly, and since you've already started that process, you're on the right track.

    0
  • ARXSystems.io Harrington

    Since this still isn't implemented for partners yet, sadly we will be looking at alternatives.

    0
  • Minetta Gould Community Team DNSFilter Staff

    Sorry to hear that, ARXSystems.io Harrington—I completely understand why this would be frustrating, especially after being told there wouldn’t be any functionality differences.

    At this time, the Data Export add-on still isn’t available through distributors like Pax8. If this feature is critical for your workflow, the best path to get access is to purchase DNSFilter direct, as noted in the article.

    0

Please sign in to leave a comment.