API (Data)

What is an API?

Short for Application Programming Interface, it's a set of rules or protocols that allow different software programs to exchange data and interact with each other.

Use this in Listly when you want to store extracted data in your own database or integrate it with Google Sheets.

TIP How to integrate with Google Sheets

Reference: Importing Collected Data in Google Sheets (API Integration)

Quick start

➡️ Select desired data extraction result ➡️ Click API button ➡️ (First time) Generate token ➡️ Get API token and API URL

---

How to check API URL

The LISTLY API can be thought of as a data pipeline for delivering extracted data to wherever you want. As shown below, every data result page has an API button, and clicking it reveals the API URL pointing to that result. Accessing that URL shows the same data you'd get in Excel. Using this feature, you can integrate with Google Sheets or write additional code to put data into your own DB.

For first-time API button users, a token generation screen appears as shown below. Generate the token with one click, return to the results page, and click the API button again.

If you've already configured it, you can check your API token in account settings.

---

API URL Types

There are 2 main API URL types:

• Latest API: Shows only the latest successfully extracted data.

• Log API: Shows all data extracted over time.

You can use both latest and log APIs on Single and Group extraction result pages. Use the stepper below to see how to check each.

1

Single Latest Data

• Single extraction latest data: Select latest in results then check API

2

Single Log Data

• Single extraction log data: Select log in results then check API

3

Group Latest Data

• Group extraction latest data: Check in Group API

4

Group Log Data

• Group extraction log data: Check in log list → select desired log data → check in Group API

Latest API response message is {"message": "Warning! Do not use this API endpoint..."}

This message appears when the web page design has changed. Data extraction still works fine, but it's a warning that the web page has changed. This is likely to occur when using the scheduler feature for automatic collection.

In principle, we recommend extracting fresh from the LISTLY extension. Since the web page design changed, creating a new reference point is better long-term. Extract new data to create a new starting point, reapply scheduler settings, and from then on you can use the latest API stably again.

If you want to ignore the warning and use existing data as is, use the address shown in the "api_endpoint" area of the message above. At this time, the latest extracted data appears, and the address shown in "api_endpoint" is the existing API address + an option to ignore warnings (&ignore_warning=y).

Introducing API Endpoints

All API URLs can be checked on the data extraction result page as shown in the image.

Type	URI	Method
Single (Latest)	www.listly.io/api/single?key=<singlekey>	GET
	Shows latest data. Used to receive latest data auto-extracted by scheduler.
Single (Log)	www.listly.io/api/single/data?key=<datakey>	GET
	Shows all log data. Used to import and review all accumulated data at once.
Group (Latest)	www.listly.io/api/group?key=<groupkey>	GET
	Shows latest group data.
Group (Log)	www.listly.io/api/group/data?key=<datakey>	GET
	Shows all data in the batch group containing the selected data. Batch increases by 1 each time new group data accumulates.

Introducing API Parameters

Single (Latest) Data API

www.listly.io/api/single?key=<singlekey>&selected=1&arrange=y&href=n&file=csv

Single (Log) Data API

www.listly.io/api/single?key=<datakey>&selected=1&arrange=y&href=n&stack=vertical&file=csv

<datakey> points to the data result that was open when the user copied the API URL. Based on the tab selected in that data, it fetches the same data from latest or log data.

Name	Type	Values	Default	Description
selected	int	1, 2, 3...	1	Position of selected tab. 1 means 1st, 2 means 2nd.
arrange	string	y, n	y	Data auto-sort option. y auto-sorts. n outputs in order of appearance from left without auto-sorting.
href	string	y, n	n	Hyperlink inclusion option. y includes hyperlinks. n doesn't include them.
stack	string	vertical, horizontal	vertical	How to stack data. vertical stacks top to bottom vertically. horizontal stacks left to right horizontally.
file	string	csv, json	json	Data output format.
from	datetime	2020-11-01T00:00		Can limit data range by datetime. from is the starting point. Expressed as %Y-%m-%dT%H:%M.
to	datetime	2020-11-11T23:59		Can limit data range by datetime. to is the ending point. Expressed as %Y-%m-%dT%H:%M.

TIP If the explanation above isn't clear, try changing option values directly in the actual API URL. You can immediately see how it changes.

---

Code Samples

Python Code Template

import requests
import csv

API_URL = "API URL"    # Enter 'API URL' of collected data
API_TOKEN = "API Token"     # Enter 'API Token' issued to your account
headers = { "Authorization": API_TOKEN }

# Load collected data
response = requests.get(url=API_URL, headers=headers)
content = response.content

# Save as CSV file
"""
If the file format loaded by API URL is JSON not CSV,
you need to use different code for that
"""
with open('./result.csv', 'wb') as f:
    f.write(content)

Python Code Actual Example

import requests
import csv

API_URL = "https://www.listly.io/api/single?key=CmKQ625S&selected=1&arrange=y&href=n&file=csv"
API_TOKEN = "BC1Sx370Yows25RxaBMuRxTuMKw3Aq0Ks"
headers = { "Authorization": API_TOKEN }

response = requests.get(url=API_URL, headers=headers)
content = response.content

with open('./result.csv', 'wb') as f:
    f.write(content)