Peaka as a Data Source

Peaka serves as a powerful data source for your data warehouse, enabling you to effortlessly create and manage projects, catalogs, and caching mechanisms. Additionally, it offers a comprehensive API that allows you to execute queries and retrieve data simultaneously, enhancing your data management capabilities.

Key Features

Project Management: Effortlessly create and manage projects, catalogs, and caching through the intuitive Peaka UI
Query Execution: Execute queries and retrieve data seamlessly using the powerful Peaka API
Advanced Caching: Implement advanced caching strategies for optimized performance
Complex Query Handling: Execute complex queries with ease
Seamless Integration: Achieve seamless integration with existing systems
Efficient Data Operations: Perform efficient data operations to enhance productivity

Step-by-Step Implementation

1. Create a Peaka Project

Before you start, you need to login to your Peaka account. If you don’t have an account, you can sign up for a free trial here.
After you login, click on the “Create New Project” button.

Enter the project name and click on the “Create Project” button.

Your project will be created and you will be redirected to the project page.

For detailed guidance, please refer to the documentation on how to create a project.

2. Create a Catalog

Once you are redirected to the project page, locate and click the plus button on the right side panel to initiate the catalog creation process. Open Connectors Modal

From the list of available options, select the catalog you wish to connect to.

Fill in the catalog name along with any other required fields to ensure proper configuration.
After entering the necessary information, click on the “Create” or “Sign In” button to finalize the process.
Your catalog will be successfully created, and you can start managing your data within it.

For detailed guidance, please refer to the documentation on how to create a catalog.

3. Caching

Once you have created the catalog, you will find the catalog table displayed in the right side panel under the “Tables” section.

Hover over the catalog item and click on the three dots icon to reveal a dropdown menu. From this menu, select the “Start Cache” option.

A modal will appear prompting you to confirm the caching action. Click on the “Start Initial Cache” button within this modal.

Note: This action may either initiate a cache job for the catalog directly or open a new modal where you can input the required fields. If prompted, fill in the necessary information and then click on the “Start Cache” button to proceed.

Cache Settings

When configuring the cache, you can control how and when your cached data is refreshed using the following settings:

Auto Update: A toggle that enables or disables automatic cache refreshes. When turned on, Peaka periodically refreshes the cache based on the update frequency you select. When turned off, the cache is not refreshed automatically, and both the Update Frequency and Cache Type options are disabled.
Update Frequency: Determines how often the cache is automatically refreshed. This option is only available when Auto Update is on. The available intervals are:
- Every 15 minutes
- Every 30 minutes
- Every 1 hour
- Every 2 hours
- Every 6 hours
- Every day
- Every week
Cache Type: Determines how the cache is refreshed on each update. This option is only available when Auto Update is on. The available types are:
- Incremental: Updates the cache with only the new or changed data since the last refresh, resulting in faster and more efficient updates.
- Full Refresh: Rebuilds the entire cache from scratch on each refresh, ensuring the cached data fully matches the source.
This will initiate the caching process for your catalog.

Choosing the Right Autoupdate Settings

Autoupdate keeps your cached data in sync with the source. Picking the wrong combination is the most common setup mistake, so use the scenarios below as a starting point:

Frequently changing data that needs to stay fresh (e.g. orders, events): enable Auto Update, choose a short Update Frequency (Every 15 or 30 minutes), and set Cache Type to Incremental. Incremental updates only pull new or changed rows, so short intervals stay inexpensive.
Large tables where you mainly need recent data: enable Auto Update with an Incremental refresh at a moderate frequency (Every 1–6 hours). Avoid frequent Full Refresh on large tables — rebuilding the whole cache repeatedly is slow and resource-heavy.
Data that must exactly match the source, including deletes and updates to old rows: use a Full Refresh. Because incremental updates only add new or changed rows, schedule a less frequent Full Refresh (Every day or Every week) to guarantee the cache fully matches the source.
Reference or rarely changing data (e.g. lookup tables): a low frequency such as Every day or Every week is usually enough.
Static data that never changes: turn Auto Update off. The cache is populated once and never refreshed automatically. You can still trigger a manual refresh later if needed.

If Auto Update is off, your cache will not refresh on its own and will gradually become stale as the source changes. Only turn it off for data you know is static, or when you intend to refresh manually.

Incremental and Full Refresh can run on independent schedules. A common pattern is a frequent Incremental update (e.g. hourly) combined with a periodic Full Refresh (e.g. weekly) to catch deletes and corrections. To configure these schedules via the API, see Cache Your Catalog.

4. Creating Partner API Key

Before starting execution of query, you need to create a Partner API key. You will use this API key to authenticate your requests to the Peaka API.
Navigate to the Developer section.
Click on the Partner API Key button to open a dialog box.
Fill in the form by entering your desired API Key Name.
Click on the Create button.
You can copy the API key from the dialog box.

For detailed guidance, please refer to the documentation on how to create a partner API key. created partner api key dialog

5. Create Your Query

Now you have created your project, API key, and catalog. Also cached your catalog.
You are now ready to create queries that allow you to extract and refine data from your catalog based on your requirements.

For detailed guidance, please refer to the documentation on how to create a query.

curl --request POST \
    --url https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries \
    --header 'Content-Type: application/json' \
    --data '{
    "displayName": "sampleQuery",
    "inputQuery": "SELECT * from \"mycatalog\".payment.customers"
}'

const options = {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: '{"displayName":"sampleQuery","inputQuery":"SELECT * from \"mycatalog\".payment.customers"}'
};

fetch('https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries', options)
    .then(response => response.json())
    .then(response => console.log(response))
    .catch(err => console.error(err));

import requests

url = "https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries"

payload = {
    "displayName": "sampleQuery",
    "inputQuery": "SELECT * from \"mycatalog\".payment.customers"
}
headers = {"Content-Type": "application/json"}

response = requests.request("POST", url, json=payload, headers=headers)

print(response.text)

<?php

$curl = curl_init();

curl_setopt_array($curl, [
    CURLOPT_URL => "https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries",
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "POST",
    CURLOPT_POSTFIELDS => "{\n  \"displayName\": \"sampleQuery\",\n  \"inputQuery\": \"SELECT * from \\\"mycatalog\\\".payment.customers\"\n}",
    CURLOPT_HTTPHEADER => [
        "Content-Type: application/json"
    ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
    echo "cURL Error #:" . $err;
} else {
    echo $response;
}

package main

import (
    "fmt"
    "strings"
    "net/http"
    "io/ioutil"
)

func main() {

    url := "https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries"

    payload := strings.NewReader("{\n  \"displayName\": \"sampleQuery\",\n  \"inputQuery\": \"SELECT * from \\\"mycatalog\\\".payment.customers\"\n}")

    req, _ := http.NewRequest("POST", url, payload)

    req.Header.Add("Content-Type", "application/json")

    res, _ := http.DefaultClient.Do(req)

    defer res.Body.Close()
    body, _ := ioutil.ReadAll(res.Body)

    fmt.Println(res)
    fmt.Println(string(body))

}

HttpResponse<String> response = Unirest.post("https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries")
    .header("Content-Type", "application/json")
    .body("{\n  \"displayName\": \"sampleQuery\",\n  \"inputQuery\": \"SELECT * from \\\"mycatalog\\\".payment.customers\"\n}")
    .asString();

{
    "id": "<string>",
    "displayName": "<string>",
    "name": "<string>",
    "inputQuery": "<string>",
    "inputQueryRefId": "<string>",
    "queryType": "<string>",
    "schedule": {
        "expression": "<string>"
    }
}

6. Run Your Query

After creating query, you can run your query.
After running query is completed, you will get the data in response. You can use this data for further processing or analysis as needed.

For detailed guidance, please refer to the documentation on how to run a query.

curl --request POST \
--url https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries/execute \
--header 'Content-Type: application/json' \
--data '{
"from": [
    {
    "catalogName": "peaka",
    "schemaName": "query",
    "tableName": "samplequery"
    }
]
}'

const options = {
    method: 'POST',
    headers: {'Content-Type': 'application/json'},
    body: '{"from":[{"catalogName":"peaka","schemaName":"query","tableName":"samplequery"}]}'
};

fetch('https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries/execute', options)
    .then(response => response.json())
    .then(response => console.log(response))
    .catch(err => console.error(err));

import requests

url = "https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries/execute"

payload = {"from": [
        {
            "catalogName": "peaka",
            "schemaName": "query",
            "tableName": "samplequery"
        }
    ]}
headers = {"Content-Type": "application/json"}

response = requests.request("POST", url, json=payload, headers=headers)

print(response.text)

<?php

$curl = curl_init();

curl_setopt_array($curl, [
    CURLOPT_URL => "https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries/execute",
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "POST",
    CURLOPT_POSTFIELDS => "{\n  \"from\": [\n    {\n      \"catalogName\": \"peaka\",\n      \"schemaName\": \"query\",\n      \"tableName\": \"samplequery\"\n    }\n  ]\n}",
    CURLOPT_HTTPHEADER => [
        "Content-Type: application/json"
    ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
    echo "cURL Error #:" . $err;
} else {
    echo $response;
}

package main

import (
    "fmt"
    "strings"
    "net/http"
    "io/ioutil"
)

func main() {

    url := "https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries/execute"

    payload := strings.NewReader("{\n  \"from\": [\n    {\n      \"catalogName\": \"peaka\",\n      \"schemaName\": \"query\",\n      \"tableName\": \"samplequery\"\n    }\n  ]\n}")

    req, _ := http.NewRequest("POST", url, payload)

    req.Header.Add("Content-Type", "application/json")

    res, _ := http.DefaultClient.Do(req)

    defer res.Body.Close()
    body, _ := ioutil.ReadAll(res.Body)

    fmt.Println(res)
    fmt.Println(string(body))

}

HttpResponse<String> response = Unirest.post("https://partner.peaka.studio/api/v1/data/projects/{projectId}/queries/execute")
    .header("Content-Type", "application/json")
    .body("{\n  \"from\": [\n    {\n      \"catalogName\": \"peaka\",\n      \"schemaName\": \"query\",\n      \"tableName\": \"samplequery\"\n    }\n  ]\n}")
    .asString();

{
  "columns": [
    {
      "catalogId": "2",
      "catalogName": "peaka",
      "schemaName": "query",
      "tableName": "samplequery",
      "columnName": "amount"
    }
  ],
  "data": [
    [
      {
        "name": "amount",
        "displayName": "amount",
        "dataType": "bigint",
        "value": "75",
        "order": 0
      }
    ],
    [
      {
        "name": "amount",
        "displayName": "amount",
        "dataType": "bigint",
        "value": "75",
        "order": 0
      }
    ],
    [
      {
        "name": "amount",
        "displayName": "amount",
        "dataType": "bigint",
        "value": "75",
        "order": 0
      }
    ]
  ]
}

7. Success

You have successfully created your project, generated your API key, established your catalog, cached the catalog, and executed your query.
This data is now ready for advanced processing and insightful analysis.
You have effectively integrated Peaka as your robust data management backend.

​Key Features

​Step-by-Step Implementation

​1. Create a Peaka Project

​2. Create a Catalog

​3. Caching

​Cache Settings

​Choosing the Right Autoupdate Settings

​4. Creating Partner API Key

​5. Create Your Query

​6. Run Your Query

​7. Success

Key Features

Step-by-Step Implementation

1. Create a Peaka Project

2. Create a Catalog

3. Caching

Cache Settings

Choosing the Right Autoupdate Settings

4. Creating Partner API Key

5. Create Your Query

6. Run Your Query

7. Success