on_page/waterfall – DataForSEO API v.3

OnPage API Waterfall

‌‌
This endpoint is designed to provide you with the page speed insights. Using this function you can get detailed information about the page loading time, time to secure connection, the time it takes to load page resources, and so on.

This feature is especially useful for creating page speed tests and other tools for checking website performance.

POST

Your account will not be charged for using this function. You can get the results of the task within the next 30 days for free.
The cost can be calculated on the Pricing page.

All POST data should be sent in the JSON format (UTF-8 encoding). The task setting is done using the POST method. When setting a task, you should send all task parameters in the task array of the generic POST array.

Description of the fields for setting a task:

Field name	Type	Description
`id`	string	ID of the task required field you can get this ID in the response of the Task POST endpoint example: “07131248-1535-0216-1000-17384017ad04”
`url`	string	page URL required field specify the pages you want to receive timing for
`tag`	string	user-defined task identifier optional field the character limit is 255 you can use this parameter to identify the task and match it with the result you will find the specified `tag` value in the `data` object of the response

‌‌‌‌‌‌
As a response of the API server, you will receive JSON-encoded data containing a tasks array with the information specific to the set tasks.

Description of the fields in the results array:

Field name	Type	Description
`version`	string	the current version of the API
`status_code`	integer	general status code you can find the full list of the response codes here Note: we strongly recommend designing a necessary system for handling related exceptional or error conditions
`status_message`	string	general informational message you can find the full list of general informational messages here
`time`	string	execution time, seconds
`cost`	float	total tasks* cost, USD*
`tasks_count`	integer	the number of tasks in the `tasks`* array*
`tasks_error`	integer	the number of tasks in the `tasks`* array returned with an error*
`tasks`	array	array of tasks
`id`	string	task identifier unique task identifier in our system in the UUID format
`status_code`	integer	status code of the task generated by DataForSEO; can be within the following range: 10000-60000 you can find the full list of the response codes here
`status_message`	string	informational message of the task you can find the full list of general informational messages here
`time`	string	execution time, seconds
`cost`	float	cost of the task, USD
`result_count`	integer	number of elements in the `result` array
`path`	array	URL path
`data`	object	contains the same parameters that you specified in the POST request
`result`	array	array of results
`crawl_progress`	string	status of the crawling session possible values: `in_progress`, `finished`
`crawl_status`	object	details of the crawling session
`max_crawl_pages`	integer	maximum number of pages to crawl indicates the `max_crawl_pages` limit you specified when setting a task
`pages_in_queue`	integer	number of pages that are currently in the crawling queue
`pages_crawled`	integer	number of crawled pages
`items_count`	integer	number of items in the results array
`items`	array	items array
`page_url`	string	URL of the page
`time_to_interactive`	integer	Time To Interactive (TTI) metric the time it takes until the user can interact with a page (in milliseconds)
`dom_complete`	integer	time to load resources the time it takes until the page and all of its subresources are downloaded (in milliseconds)
`connection_time`	integer	time to connect to a server the time it takes until the connection with a server is established (in milliseconds)
`time_to_secure_connection`	integer	time to establish a secure connection the time it takes until the secure connection with a server is established (in milliseconds)
`request_sent_time`	integer	time to send a request to a server the time it takes until the request to a server is sent (in milliseconds)
`waiting_time`	integer	time to first byte (TTFB) in milliseconds
`download_time`	integer	time it takes for a browser to receive a response (in milliseconds)
`duration_time`	integer	total time it takes until a browser receives a complete response from a server (in milliseconds)
`fetch_start`	integer	time to start downloading the HTML resource the amount of time the browser needs to start downloading a page
`fetch_end`	integer	time to complete downloading the HTML resource the amount of time the browser needs to complete downloading a page
`resources`	array	resource-specific timing contains separate arrays with timing for each resource found on the page
`resource_type`	string	type of the returned resource
`url`	string	resource URL
`initiator`	string	resource initiator
`duration_time`	integer	total time it takes until a browser receives a complete response from a server (in milliseconds)
`fetch_start`	integer	time to start downloading the resource the amount of time the browser needs to start downloading a resource
`fetch_end`	integer	time to complete downloading the resource the amount of time the browser needs to complete downloading a resource
`location`	object	location of the resource in the document parameters defining the location of the specific resource within the document’s HTML
`line`	integer	line number the number of the line on which the resource is located
`offset_left`	integer	position in line the number of line characters before the resource; sometimes referred to as column Note: counts from 1, i.e. if the resource doesn’t have any characters to the left, the value will be 1
`offset_top`	integer	position in the document the total number of characters between the resource and the top of HTML
`is_render_blocking`	boolean	indicates whether the resource blocks rendering

‌‌

Instead of ‘login’ and ‘password’ use your credentials from https://app.dataforseo.com/api-access

# Instead of 'login' and 'password' use your credentials from https://app.dataforseo.com/api-access 
login="login" 
password="password" 
cred="$(printf ${login}:${password} | base64)" 
curl --location --request POST "https://api.dataforseo.com/v3/on_page/waterfall" 
--header "Authorization: Basic ${cred}"  
--header "Content-Type: application/json" 
--data-raw "[
  {
    "id": "08101204-0696-0216-0000-644a7b21a48a",
    "url": "https://dataforseo.com/tag/broken-links"
  }
]"

<?php
// You can download this file from here https://cdn.dataforseo.com/v3/examples/php/php_RestClient.zip
require('RestClient.php');
$api_url = 'https://api.dataforseo.com/';
// Instead of 'login' and 'password' use your credentials from https://app.dataforseo.com/api-access
$client = new RestClient($api_url, null, 'login', 'password');

$post_array = array();
// simple way to get a result
$post_array[] = array(
   "id" => "07281559-0695-0216-0000-c269be8b7592",
   "url" => "https://dataforseo.com/tag/broken-links"
);
try {
   // POST /v3/on_page/waterfall
   // the full list of possible parameters is available in documentation
   $result = $client->post('/v3/on_page/waterfall', $post_array);
   print_r($result);
   // do something with post result
} catch (RestClientException $e) {
   echo "n";
   print "HTTP code: {$e->getHttpCode()}n";
   print "Error code: {$e->getCode()}n";
   print "Message: {$e->getMessage()}n";
   print  $e->getTraceAsString();
   echo "n";
}
$client = null;
?>

const post_array = [];

post_array.push({
  "id": "07281559-0695-0216-0000-c269be8b7592",
  "url": "https://dataforseo.com/tag/broken-links"
});

const axios = require('axios');

axios({
  method: 'post',
  url: 'https://api.dataforseo.com/v3/on_page/waterfall',
  auth: {
    username: 'login',
    password: 'password'
  },
  data: post_array,
  headers: {
    'content-type': 'application/json'
  }
}).then(function (response) {
  var result = response['data']['tasks'];
  // Result data
  console.log(result);
}).catch(function (error) {
  console.log(error);
});

from random import Random
from client import RestClient
# You can download this file from here https://api.dataforseo.com/v3/_examples/python/_python_Client.zip
client = RestClient("login", "password")

post_data = dict()
# simple way to get a result
post_data[len(post_data)] = dict(
    id="07281559-0695-0216-0000-c269be8b7592",
    url="https://dataforseo.com/tag/broken-links"
)
# POST /v3/on_page/waterfall
# the full list of possible parameters is available in documentation
response = client.post("/v3/on_page/waterfall", post_data)
# you can find the full list of the response codes here https://docs.dataforseo.com/v3/appendix/errors
if response["status_code"] == 20000:
    print(response)
    # do something with result
else:
    print("error. Code: %d Message: %s" % (response["status_code"], response["status_message"]))

using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;

namespace DataForSeoDemos
{
    public static partial class Demos
    {
        public static async Task on_page_waterfall()
        {
            var httpClient = new HttpClient
            {
                BaseAddress = new Uri("https://api.dataforseo.com/"),
                // Instead of 'login' and 'password' use your credentials from https://app.dataforseo.com/api-access
                DefaultRequestHeaders = { Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(Encoding.ASCII.GetBytes("login:password"))) }                
            };
            var postData = new List<object>();
            // simple way to get a result
            postData.Add(new
            {
                id = "07281559-0695-0216-0000-c269be8b7592",
                url = "https://dataforseo.com/tag/broken-links"
            });
            // POST /v3/on_page/waterfall
            // the full list of possible parameters is available in documentation
            var taskPostResponse = await httpClient.PostAsync("/v3/on_page/waterfall", new StringContent(JsonConvert.SerializeObject(postData)));
            var result = JsonConvert.DeserializeObject<dynamic>(await taskPostResponse.Content.ReadAsStringAsync());
            // you can find the full list of the response codes here https://docs.dataforseo.com/v3/appendix/errors
            if (result.status_code == 20000)
            {
                // do something with result
                Console.WriteLine(result);
            }
            else
                Console.WriteLine($"error. Code: {result.status_code} Message: {result.status_message}");

            Console.ReadLine();
        }
    }
}

The above command returns JSON structured like this:

{
  "version": "0.1.20221214",
  "status_code": 20000,
  "status_message": "Ok.",
  "time": "0.1119 sec.",
  "cost": 0,
  "tasks_count": 1,
  "tasks_error": 0,
  "tasks": [
    {
      "id": "02251612-1535-0216-0000-26ca912e97f9",
      "status_code": 20000,
      "status_message": "Ok.",
      "time": "0.0594 sec.",
      "cost": 0,
      "result_count": 1,
      "path": [
        "v3",
        "on_page",
        "waterfall"
      ],
      "data": {
        "api": "on_page",
        "function": "waterfall",
        "url": "https://dataforseo.com/help-center",
        "target": "dataforseo.com",
        "max_crawl_pages": 100,
        "load_resources": true
      },
      "result": [
        {
          "crawl_progress": "finished",
          "crawl_status": {
            "max_crawl_pages": 100,
            "pages_in_queue": 0,
            "pages_crawled": 100
          },
          "items_count": 1,
          "items": [
            {
              "page_url": "https://dataforseo.com/help-center",
              "time_to_interactive": 644,
              "dom_complete": 644,
              "connection_time": 13,
              "time_to_secure_connection": 18,
              "request_sent_time": 0,
              "waiting_time": 0,
              "download_time": 5,
              "duration_time": 36,
              "fetch_start": 0,
              "fetch_end": 36,
              "resources": [
                {
                  "resource_type": "stylesheet",
                  "url": "https://dataforseo.com/wp-content/plugins/dataforseo-kb/assets/css/grid.min.css?ver=5.8.6",
                  "initiator": "(index)",
                  "duration_time": 27,
                  "fetch_start": 36,
                  "fetch_end": 63,
                  "location": {
                    "line": 50,
                    "offset_left": 1,
                    "offset_top": 6487
                  },
                  "is_render_blocking": false
                },
                {
                  "resource_type": "stylesheet",
                  "url": "https://dataforseo.com/wp-content/plugins/dataforseo-kb/assets/css/search.min.css?ver=5.8.6",
                  "initiator": "(index)",
                  "duration_time": 29,
                  "fetch_start": 36,
                  "fetch_end": 65,
                  "location": {
                    "line": 49,
                    "offset_left": 1,
                    "offset_top": 6315
                  },
                  "is_render_blocking": false
                },
                {
                  "resource_type": "stylesheet",
                  "url": "https://dataforseo.com/wp-content/plugins/dataforseo-reload-robots/assets/css/style.min.css?ver=5.8.6",
                  "initiator": "(index)",
                  "duration_time": 38,
                  "fetch_start": 36,
                  "fetch_end": 74,
                  "location": {
                    "line": 51,
                    "offset_left": 1,
                    "offset_top": 6655
                  },
                  "is_render_blocking": false
                },
                {
                  "resource_type": "stylesheet",
                  "url": "https://dataforseo.com/wp-content/plugins/dataforseo-updates/assets/css/style.min.css?ver=5.8.6",
                  "initiator": "(index)",
                  "duration_time": 30,
                  "fetch_start": 36,
                  "fetch_end": 66,
                  "location": {
                    "line": 52,
                    "offset_left": 1,
                    "offset_top": 6838
                  },
                  "is_render_blocking": false
                },
                {
                  "resource_type": "stylesheet",
                  "url": "https://dataforseo.com/wp-content/plugins/wp-video-lightbox/css/prettyPhoto.css?ver=5.8.6",
                  "initiator": "(index)",
                  "duration_time": 32,
                  "fetch_start": 36,
                  "fetch_end": 68,
                  "location": {
                    "line": 53,
                    "offset_left": 1,
                    "offset_top": 7023
                  },
                  "is_render_blocking": false
                },
                {
                  "resource_type": "stylesheet",
                  "url": "https://dataforseo.com/wp-content/plugins/wp-video-lightbox/wp-video-lightbox.css?ver=5.8.6",
                  "initiator": "(index)",
                  "duration_time": 29,
                  "fetch_start": 63,
                  "fetch_end": 92,
                  "location": {
                    "line": 54,
                    "offset_left": 1,
                    "offset_top": 7202
                  },
                  "is_render_blocking": false
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}