Log in to ask David Esner questions publicly or anonymously.
Hello everyone, we have new Pinterest Ads data source component available in beta and we are looking for testers. It allows creating and getting performance reports across multiple ad accounts or automating retrieval of Custom reports defined via Pinterest UI.
It is available under component ID kds-team.ex-pinterest If you're interested let me know here in the thread!
We're excited to announce the release our new BigCommerce data source ! BigCommerce is a powerful e-commerce platform that allows businesses to sell their products online. The component supports endpoints for orders, customers, brands, and products, and makes it easy to extract data from your Big Commerce store. Plus, it supports incremental loads, so you can keep your data up to date without having to extract everything each time.
If you have any questions or feedback, please do not hesitate to contact us.
We are pleased to announce the release of a new data source component, Microsoft Advertising (Bing Ads), which allows you to easily retrieve data from all report types available on the Microsoft Advertising platform. This new component replaces the old public Bing Ads extractor (esnerda.ex-bingads), which will be deprecated and no longer supported. However, the old component will still be functional.
The new Microsoft Advertising component comes with many user interface and user experience improvements. It does not require users to provide a Developer Token anymore, making it easier to use. Moreover, the report data can be downloaded incrementally, allowing for automated retrieval of increments. Users can either define a custom report or use one of the predefined presets.
Due to some structural changes, there is no automated migration available at this time. However, the result data should have the same structure as in the old component. If you require assistance with migration, please do not hesitate to contact our support team.
The new component allows you to define custom reports or use on of the presets available:
We hope that the new Microsoft Advertising component will improve your experience. If you have any questions or feedback, please do not hesitate to contact us.
We are excited to announce the release of the Rename Columns Processor! This processor allows you to rename columns for specific tables or multiple tables defined by a wildcard. It takes all tables in the in/tables folder, renames specified columns in specified tables, and moves the result to out/tables. All other tables that are not specified or columns that are not specified will be moved unchanged. The in/files folder is also moved unchanged. Manifest files are respected and transferred/modified as well.
The typical usecase would be to rename only one specific column so it matches the requirements of Keboola Storage, e.g. when it's too long or contains forbidden characters and you want to control the resulting name. This is more suitable than Create Manifest processor for these purposes because you can selectively choose only specific tables.
Hi all, we have just published a new processor Drop Columns that is useful when you want to exclude some columns from your incoming sources (e.g. PIIs that you do not need).
To use this you can just add this processor into the after section. The bellow configuration excludes columns Location and Type from all tables that start with test prefix, e.g. test.csv:
{
"definition": {
"component": "kds-team.processor-drop-columns"
},
"parameters": {
"table_mapping": {
"test*": [
"Location",
"Type"
]
}
}
}Note that the table name must match what the component output in the data/out/tables . The easiest way to see that is to run a debug job
Hi all. We would like to create a new version of the BingAds (MS Advertising) extractor that is more stable, with improved UI/UX and some additional functionality. Is there anyone who uses it frequently and has some suggestions on what we may do differently? We are also in search of some testers that would help us to tune the new version on live data.
Thank you
Hello community! We are about to start work on new Huspot (CRM, Sales) Extractor V2 and we'd like your input on what can be improved. Now is the chance to tell us what features you're missing or what can be done differently. Here's what we plan to change so far:
Anything we missed? Also if you'd like to join testing of the new connector let us know.
Hi all, some of you are already using the Google Apps script that runs in Google sheet and creates option for the user to trigger a Keboola Orchestration directly from the sheet. We have created and updated version of that which supports the new queue, e.g. Orchestrations v2 or Flows.
Here's a guide on how to deploy it in you sheet:
Code.gs (replace the existing content)Keboola will appear.Run orchestration you will be asked to provide url to your orchestration + Storage token (that will be saved for next run)
/*
* @OnlyCurrentDoc
*/
function runKeboola(skip_wait) {
try {
var ui = SpreadsheetApp.getUi();
var properties = PropertiesService.getDocumentProperties()
var orch_url = properties.getProperty("orch_url")
if (orch_url === null) {
var orch_url = ui.prompt("Enter the whole orchestration URL from Keboola Connection").getResponseText();
if (orch_url === "") return;
properties.setProperty("orch_url", orch_url);
var reg_exp1 = new RegExp("(.+(orchestrations-v2|flows)\/)(\\d*)");
var id = reg_exp1.exec(orch_url)[3];
ui.alert("Orchestration ID check: " + id);
properties.setProperty("Parsed Orchestration ID", id);
var reg_exp2 = new RegExp("(?<=connection.).*keboola.com");
var region = reg_exp2.exec(orch_url);
ui.alert("Parsed Base URL check: " + region);
properties.setProperty("region", region);
} else {
orch_url = properties.getProperty("orch_url");
var reg_exp1 = new RegExp("(.+(orchestrations-v2|flows)\/)(\\d*)");
var id = reg_exp1.exec(orch_url)[3];
properties.setProperty("Parsed Orchestration ID", id);
var reg_exp2 = new RegExp("(?<=connection.).*keboola.com");
var region = reg_exp2.exec(orch_url);
}
var token = properties.getProperty("token")
if (token === null) {
var token = ui.prompt("Enter API token").getResponseText();
if (token === "") return;
properties.setProperty("token", token);
} else {
token = properties.getProperty("token");
}
var isFinished = false;
var res = '';
var url = Utilities.formatString("https://queue.%s/jobs", region);
var data = {
"component": "keboola.orchestrator",
"mode": "run",
"config": id
}
var params = {
method: "POST",
'contentType': 'application/json',
// Convert the JavaScript object to a JSON string.
'payload' : JSON.stringify(data),
headers: {
"x-storageapi-token": token
}
}
Logger.log(params);
var res = JSON.parse(UrlFetchApp.fetch(url, params).getContentText());
var jobID = res.id;
while (isFinished == false){
var url = Utilities.formatString("https://queue.%s/jobs/" + jobID, region);
var params = {
method: "GET",
contentType: "application/json",
headers: {
"x-storageapi-token": token
}
}
var res = JSON.parse(UrlFetchApp.fetch(url, params).getContentText());
isFinished = Boolean(res.isFinished);
var htmlOutput = buildStatusModal(res.status.toUpperCase());
if (prevStatus != res.status.toUpperCase()) {
var prevStatus = res.status.toUpperCase();
SpreadsheetApp.getUi().showModelessDialog(htmlOutput, 'Keboola Orchestration');
}
;
if (skip_wait && res.status.toUpperCase()==="PROCESSING") {
isFinished = true;
Utilities.sleep(1000);
ui.alert("Orchestration started successfuly");
}
else {
Utilities.sleep(3000);
}
}
} catch (e) {
if (e.message.indexOf("has 1 waiting job")>-1) {
ui.alert("Another orchestration already waiting");
}
else {
ui.alert("Error: "+ e.message);
}
}
}
function buildStatusModal(stage) {
var perc = 0;
switch (stage) {
case "WAITING":
perc = 25;
break;
case "PROCESSING":
perc = 75;
break;
case "SUCCESS":
perc = 100;
break;
default:
perc = 0;
}
var html =
'<html>' +
'<head><link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous"></head>' +
'<body>' +
'<div class="progress">' +
'<div class="progress-bar progress-bar-striped progress-bar-animated" role="progressbar" aria-valuenow="' + perc + '" aria-valuemin="0" aria-valuemax="100" style="width: ' + perc + '%">' + stage + '</div>' +
'</div>' +
' <script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>' +
'<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>' +
'</body>';
var htmlOutput = HtmlService
.createHtmlOutput(html)
.setWidth(300)
.setHeight(50);
return htmlOutput
}
function reset() {
var properties = PropertiesService.getDocumentProperties();
var ui = SpreadsheetApp.getUi();
properties.deleteAllProperties();
ui.alert("Properties reset");
}
function runAndWait() {
runKeboola(false);
}
function runAndForget() {
runKeboola(true);
}
function parse_url() {
try {
var ui = SpreadsheetApp.getUi();
var properties = PropertiesService.getDocumentProperties();
var my_url = ui.prompt("Enter orchestration url").getResponseText();
ui.alert("Check: " + my_url);
var reg_exp1 = new RegExp("(?<=orchestrations-v2\/)\\d*");
var tmp_id = reg_exp1.exec(my_url);
ui.alert("ID: " + tmp_id);
var reg_exp2 = new RegExp("(?<=connection.).*keboola.com");
var tmp_url = reg_exp2.exec(my_url);
ui.alert("Base URL: " + tmp_url);
} catch (e) {
ui.alert("Error: "+ e.message);
}
}
function onOpen() {
SpreadsheetApp.getUi()
.createMenu("Keboola")
.addItem("Run Orchestration", "runAndWait")
.addItem("Reset Settings", "reset")
// .addItem("URL Parse Test", "parse_url")
.addToUi()
}Hello community, we have a new Zoho CRM extractor component ready for Beta testing. Is there anyone who would be willing to be part of the testing?
Your help will be much appreciated and we'll be able to tune it to your liking.
Thank you!
Hi everyone, for all Czechs around you may find useful this new processor we recently published. It allows you to process PDF account statements issued by Komercni Banka and store them in tabular form in Storage! It works best in combination with the IMAP Email Content datasource. Just schedule the statements to your email and set up the configuration.
