{"id":3556,"date":"2021-07-27T09:56:24","date_gmt":"2021-07-27T09:56:24","guid":{"rendered":"https:\/\/wiki.thomasandsofia.com\/?p=3556"},"modified":"2022-07-13T15:32:50","modified_gmt":"2022-07-13T15:32:50","slug":"zenoss-streaming-data-training","status":"publish","type":"post","link":"https:\/\/wiki.thomasandsofia.com\/?p=3556","title":{"rendered":"Zenoss Streaming Data Training"},"content":{"rendered":"<h1>Resources<\/h1>\n<ul>\n<li><a href=\"\/streaming-data-policy\/\">Streaming Data Policy<\/a><\/li>\n<li>Day 1: s3.us-east-2.amazonaws.com\/s3.thomasandsofia.com\/zenoss\/training\/streaming-data\/20210303-Streaming_Data_to_Zenoss_Cloud-D1.mp4<\/li>\n<li>Day 2: s3.us-east-2.amazonaws.com\/s3.thomasandsofia.com\/zenoss\/training\/streaming-data\/20210304-Streaming_Data_to_Zenoss_Cloud-D2.mp4<\/li>\n<\/ul>\n<h1>Day 1<\/h1>\n<h2>Why Stream Data<\/h2>\n<ul>\n<li>Short lived applications or services (docker containers)\n<ul>\n<li>Container based services<\/li>\n<li>Microservices<\/li>\n<li>Serverless functions<\/li>\n<\/ul>\n<\/li>\n<li>Need to Push data from application or service vs. having them Pulled<\/li>\n<li>Data that is not exactly &#8220;Metrics&#8221; but want displayed on a dashboard\n<ul>\n<li>Business or Costs data<\/li>\n<\/ul>\n<\/li>\n<li>Events<\/li>\n<\/ul>\n<h2>Streaming Data Exporters and Agents<\/h2>\n<p>0:05:00<\/p>\n<ul>\n<li>OpenCensus\n<ul>\n<li>Set of libraries in multiple languages that collect application metrics and transfer to backend of your choice.<\/li>\n<li>Zenoss provides OC exporters for Go, JavaScript and Python<\/li>\n<\/ul>\n<\/li>\n<li>Prometheus\n<ul>\n<li>Requires the Zenoss Agent that acts as a remote database for Prometheus data.<\/li>\n<li>Agent converts data to metric and model data then streams to Zenoss Cloud<\/li>\n<li>Agent is deployed into a Kubernetes cluster as a Helm chart packaging method<\/li>\n<\/ul>\n<\/li>\n<li>StatsD\n<ul>\n<li>Is both simple protocol and set of software tools to collect, aggregate and publish metrics as defined by your application.<\/li>\n<li>Written in Go\n<ul>\n<li>Support for many languages<\/li>\n<li>Application needs to support the protocol<\/li>\n<\/ul>\n<\/li>\n<li>Zenoss Docs: <a href=\"https:\/\/help.zenoss.com\/docs\/streaming-data-to-zenoss-cloud\/using-a-statsd-agent-to-stream-data\" target=\"_blank\" rel=\"noopener\">https:\/\/help.zenoss.com\/docs\/streaming-data-to-zenoss-cloud\/using-a-statsd-agent-to-stream-data<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-3557\" src=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview.png\" alt=\"\" width=\"1218\" height=\"411\" srcset=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview.png 1218w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview-300x101.png 300w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview-1024x346.png 1024w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview-768x259.png 768w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/StatsD-Overview-150x51.png 150w\" sizes=\"auto, (max-width: 1218px) 100vw, 1218px\" \/><\/a><\/p>\n<ul>\n<li>Grow your own\n<ul>\n<li>Can create an agent or enable an application to send metrics directly to Zenoss cloud in any language that supports creating a secure connection to the API server using HTTPS<\/li>\n<li>We&#8217;ll demonstrate using shell scripts and Python programs<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>Zenoss API Keys<\/h2>\n<p>0:10:00<\/p>\n<ul>\n<li>Login: <a href=\"https:\/\/train.zenoss.io\" target=\"_blank\" rel=\"noopener\">https:\/\/train.zenoss.io<\/a><\/li>\n<li>Streaming Data keys enable agents and applications to publish data directly to the Zenoss Cloud.\n<ul>\n<li>Data pushed to cloud.\u00a0 mono-directional<\/li>\n<\/ul>\n<\/li>\n<li>Collection Zone keys enable you to automate tasks using the Collection Zone JSON API\n<ul>\n<li>Uses Routers and Methods to perform management functions. bi-directional\n<ul>\n<li>Add devices, users, change properties, etc.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>Must have Manager or Key Administrator roles to generate Streaming Data API keys\n<ul>\n<li>https:\/\/help.zenoss.com\/doxs\/administering-zenoss-cloud\/enabling-user-access\/roles-and-permissions<\/li>\n<\/ul>\n<\/li>\n<li>API keys are grouped by client\n<ul>\n<li>Client is an organizing structure<\/li>\n<\/ul>\n<\/li>\n<li>For Streaming Data API keys, specify the client name (often the platform or application being monitored) but can be any string.<\/li>\n<li>Each client may have multiple API keys\n<ul>\n<li>Example: A given set of microservices that handle online processing, you might create a clinet named &#8220;Order Processing&#8221; and create an API key for each microservice.<\/li>\n<li>API Keys provide separate credentials for each user, service or function in a client environment.<\/li>\n<li>Using separate keys allows you to delete a key for a specific service that has been decommissioned, compromised, etc. without affecting the other services under that client.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h3>Creating the API Client<\/h3>\n<ul>\n<li>Zenoss Cloud &gt; Admin &gt; API Clients &gt; [ + ADD CLIENT ]\n<ul>\n<li>mYz3n0s%_fAk3ApIk3Yg035h3Re_d0noTfoRGeT<\/li>\n<\/ul>\n<\/li>\n<li>Once a key is generated, the user must save.\u00a0 Zenoss does not save the key after it has been displayed.<\/li>\n<\/ul>\n<h2>Using the Zenoss Cloud API<\/h2>\n<ul>\n<li>Uses HTTPS (Port 443 must be open outbound)<\/li>\n<li>Requires a valid APO key to be included in the HTTP headers\n<ul>\n<li>&#8220;zenoss-api-key:mYz3n0s%_fAk3ApIk3Yg035h3Re_d0noTfoRGeT<\/li>\n<\/ul>\n<\/li>\n<li>All incoming requests are sent to the same endpoint\n<ul>\n<li>https:\/\/api.zenoss.io\/<\/li>\n<li>The API key identifies the Zenoss Cloud instance where the key was generated.<\/li>\n<li>This eliminates the need for a separate API URL per organization<\/li>\n<\/ul>\n<\/li>\n<li>Expects the body of the request to be in JSON format<\/li>\n<\/ul>\n<h3>API URLs<\/h3>\n<ul>\n<li>Depending on the operation, the full path will vary\n<ul>\n<li>Model data: https:\/\/api.zenoss.ip\/v1\/data-receiver\/models\n<ul>\n<li>Receives model data<\/li>\n<\/ul>\n<\/li>\n<li>Metric data: https:\/\/api.zenoss.ip\/v1\/data-receiver\/metrics\n<ul>\n<li>Receives metric data<\/li>\n<\/ul>\n<\/li>\n<li>Data Dictionary: https:\/\/api.zenoss.ip\/v1\/dictionary\/metrics\n<ul>\n<li>Sends and Receives metric dictionary data<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>The HTTP request method (verb) may vary\n<ul>\n<li>Model and Metric data use POST<\/li>\n<li>Data Dictionary methods may use GET, POST, PATCH or DELETE<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h1>Models and Metrics, Dimensions and Metadata<\/h1>\n<p>0:37:00<\/p>\n<h2>Models and Entities<\/h2>\n<p>Official Entities Documentation: <a href=\"https:\/\/docs.zenoss.io\/api\/receiver\/understanding-entities.html#reserved-keys\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.zenoss.io\/api\/receiver\/understanding-entities.html#reserved-keys<\/a><\/p>\n<ul>\n<li>Entities define the source of Metrics<\/li>\n<li>Before data can be sent, the data needs to be associated to an Entity\n<ul>\n<li>This requirement will eventually be removed.<\/li>\n<\/ul>\n<\/li>\n<li>Entities include\n<ul>\n<li>Devices<\/li>\n<li>Containers<\/li>\n<li>Applications, Services and Microservices<\/li>\n<li>Kubernetes clusters, namespaces, nodes and pods<\/li>\n<\/ul>\n<\/li>\n<li>Entities are defined by their Model\n<ul>\n<li>Models define the characteristics of an Entity<\/li>\n<\/ul>\n<\/li>\n<li>Typically, streaming agents will publish their models at start up then publish their metrics on regular polling cycles (60 seconds for example)<\/li>\n<\/ul>\n<h2>Dimensions<\/h2>\n<ul>\n<li>Required for models and metrics<\/li>\n<li>Provide context for, describe, or vategorize a model or metric<\/li>\n<li>Consist of a set of one or more name-value pairs<\/li>\n<\/ul>\n<pre>   \"dimensions\": {\r\n      \"source\": \"zennycorp.com\",\r\n      \"source-host\": \"web-02.zennycorp.com\",\r\n      \"source-type\": \"com.zenoss.training\"\r\n   }<\/pre>\n<ul>\n<li>If there is no matching <strong>policy<\/strong>, a metric is associated with an entity only if its dimensions fields <strong>exactly<\/strong> match the dimensions fields of the entity&#8217;s model.<\/li>\n<li>Dimensions fields should be thought of an <strong>invariant<\/strong>.\u00a0 If they are changed, the resulting model or metric then refers to a different entity.<\/li>\n<li>You may use any dimension field names you wish to define your entity, as long an any metrics you send have the same exact dimension field names.<\/li>\n<\/ul>\n<h3>Models &#8211; Metrics example<\/h3>\n<p><strong>Models<\/strong><\/p>\n<pre>{\r\n \"models\": [\r\n  {\r\n   \"timestamp\": 1593883448000,\r\n   \"dimensions\": {\r\n    \"source\": \"zennycorp.com\",\r\n    \"source-host\": \"web-02.zennycorp.com\",\r\n    \"source-type\": \"com.zenoss.training\"\r\n   },\r\n   \"metadataFields\": {\r\n    \"name\": \"web-02.zennycorp.com\"\r\n   }\r\n  }\r\n ]\r\n}<\/pre>\n<p><strong>Metrics<\/strong><\/p>\n<pre>{\r\n \"metrics\": [\r\n  {\r\n   \"metric\": \"resources_load_ave_5min\",\r\n   \"timestamp\": 1593883448000,\r\n   \"value\": 4.03,\r\n   \"dimensions\": {\r\n    \"source\": \"zennycorp.com\",\r\n    \"source-host\": \"web-02.zennycorp.com\",\r\n    \"source-type\": \"com.zenoss.training\"\r\n   }\r\n  }\r\n ]\r\n}<\/pre>\n<h3>Dimensions Fields<\/h3>\n<h4>source<\/h4>\n<ul>\n<li>The &#8220;source&#8221; field groups entities and metrics together for display in Zenoss Cloud dashboards<\/li>\n<li>When you create a dashboard, you select one or more source values.\u00a0 Entities that match one of those values become the scope for that dashboard.<\/li>\n<li>Only metrics belonging to entities within a dashboard&#8217;s scope are available to that dashboard&#8217;s tiles.<\/li>\n<li>Note: The &#8220;source&#8221; field can either be a dimension or part of the metadata.<\/li>\n<\/ul>\n<h4>source-type<\/h4>\n<ul>\n<li>Identifies the general type of data of the models and metrics in which it appears.\u00a0 This is often the name of the agent generating the data\n<ul>\n<li>Example: The Zenoss Cloud Kubernetes agent specifies a source type of &#8220;zenoss.agent.kubernetes&#8221;<\/li>\n<\/ul>\n<\/li>\n<li>The source field can be thought of an in instance of the source-type\n<ul>\n<li>Example: The Zenoss Cloud Kubernetes agent uses the cluster name as the value for the source.<\/li>\n<\/ul>\n<\/li>\n<li>source-type can be either a dimensions or metadata field.<\/li>\n<\/ul>\n<h3>Using Multiple Metrics example<\/h3>\n<h4>Same dimension (same model), multiple metrics.<\/h4>\n<ul>\n<li>Metric name changes (yellow)<\/li>\n<li>Dimensions fields stay the same (blue)<\/li>\n<\/ul>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-3566\" src=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics.png\" alt=\"\" width=\"1670\" height=\"743\" srcset=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics.png 1670w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics-300x133.png 300w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics-1024x456.png 1024w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics-768x342.png 768w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics-1536x683.png 1536w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multiplemetrics-150x67.png 150w\" sizes=\"auto, (max-width: 1670px) 100vw, 1670px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multimetricgraph.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-3567\" src=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multimetricgraph.png\" alt=\"\" width=\"1019\" height=\"696\" srcset=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multimetricgraph.png 1019w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multimetricgraph-300x205.png 300w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multimetricgraph-768x525.png 768w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multimetricgraph-150x102.png 150w\" sizes=\"auto, (max-width: 1019px) 100vw, 1019px\" \/><\/a><\/p>\n<h4>Same Metric, multiple Entities<\/h4>\n<ul>\n<li>Metric name stays the same (yellow)<\/li>\n<li>Different Metric Values<\/li>\n<li>Different Dimensions (blue)<\/li>\n<\/ul>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-3569\" src=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities.png\" alt=\"\" width=\"1665\" height=\"737\" srcset=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities.png 1665w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities-300x133.png 300w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities-1024x453.png 1024w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities-768x340.png 768w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities-1536x680.png 1536w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientities-150x66.png 150w\" sizes=\"auto, (max-width: 1665px) 100vw, 1665px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientitiesgraph.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-3570\" src=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientitiesgraph.png\" alt=\"\" width=\"1020\" height=\"693\" srcset=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientitiesgraph.png 1020w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientitiesgraph-300x204.png 300w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientitiesgraph-768x522.png 768w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/multientitiesgraph-150x102.png 150w\" sizes=\"auto, (max-width: 1020px) 100vw, 1020px\" \/><\/a><\/p>\n<h2>Metadata Fields<\/h2>\n<p>0:50:00<\/p>\n<ul>\n<li>Metadata consists of any additional model or metric fields that are not dimensions.\u00a0 These are attributes that are not required to uniquely identify the model or metric.<\/li>\n<li>The <strong>Model<\/strong> should always include a name metadata field.\u00a0 Think of it as the title or label for the entity the model describes<\/li>\n<li>Metadata fields may change over time<\/li>\n<li>If the <strong>source<\/strong> field is not required for uniqueness, it can be specified as a metadata field instead of a dimension field.<\/li>\n<\/ul>\n<h2>Model Fields<\/h2>\n<p>0:54:00<\/p>\n<ul>\n<li>timestamp (required): The date and time the model was generated as milliseconds since the epoch (Jan. 1, 1970 at 00:00:00 UTC)<\/li>\n<li>dimensions (required): A set of name-value pairs that define the entity represended by the model.<\/li>\n<li>metadataFields (required): A set of name-value pairs that provide additional information about the entity.\n<ul>\n<li>The <strong>name<\/strong> metadata field should always be present.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>Metric Fields<\/h2>\n<ul>\n<li>timestamp (required)<\/li>\n<li>metric (required): Name of the metric<\/li>\n<li>value (required): Value of the metric.\u00a0 MUST BE NUMERIC<\/li>\n<li>dimensions (required)<\/li>\n<li>metadataFields (Not required)<\/li>\n<\/ul>\n<h1>Sending Data to the API<\/h1>\n<p>Personal note: Working from my user folder on cloudse<\/p>\n<p>0:57:00<\/p>\n<h2>Getting setup in PuTTY<\/h2>\n<p>who cares<\/p>\n<h2>Using Linux<\/h2>\n<ul>\n<li>Use<\/li>\n<\/ul>\n<h2>JSON Overview<\/h2>\n<p>1:08:00<\/p>\n<ul>\n<li>Easy for humans to read and write<\/li>\n<li>Easy for machines to parse<\/li>\n<li>&#8220;Name&#8221; and &#8220;Strings&#8221; must be wrapped in double quotes.<\/li>\n<li>Arrays: [ value1, &#8220;value2&#8221;, value3]<\/li>\n<li>Numbers: Integers, floats, E-Notation<\/li>\n<li>true, false, null: Must all be written in lower case.<\/li>\n<li>Anything within curly brackets is an Object.\n<ul>\n<li>Objects may contain\n<ul>\n<li>lists []<\/li>\n<li>dictionaries {&#8220;name1&#8221;: value1, &#8220;name2&#8221;: &#8220;value2&#8221;}<\/li>\n<li>Other objects<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>Anatomy of an API Call<\/h2>\n<ul>\n<li>Host: api.zenoss.io<\/li>\n<li>Verb (-? ): POST, GET, etc.<\/li>\n<li>content-type: application\/json<\/li>\n<li>zenoss-api-key: y0urZ3n0ssApiKeyH3r3<\/li>\n<li>Data (-d ): JSON wrapped in single quotes so Names and Strings can be in double.<\/li>\n<li>if call is successful, returns 200<\/li>\n<\/ul>\n<h3>Using the API key with Linux<\/h3>\n<ul>\n<li>Copy the key to ~\/.zenoss.key<\/li>\n<li><code>chmod 600 ~\/.zenoss.key<\/code><\/li>\n<\/ul>\n<h3>Get the Date<\/h3>\n<ul>\n<li><code>TIME_NOW_MS=$(date +%s000)<\/code><\/li>\n<\/ul>\n<h1>Using Bash and Curl<\/h1>\n<h3>Creating a Simple JSON Model File (Exercise 6.4a &amp; b)<\/h3>\n<pre>cat &gt; simple-model.json &lt;&lt; EoT\r\n{\r\n  \"models\": [\r\n    {\r\n      \"timestamp\": $(date +s000),\r\n      \"dimensions\": {\r\n        \"source\": \"$USER\",\r\n        \"source-type\": \"com.zenoss.training\"\r\n      },\r\n      \"metadataFields\": {\r\n        \"name\": \"${USER}-$(hostname -f)\"\r\n      }\r\n    }\r\n  ]\r\n}\r\nEoT<\/pre>\n<h2>Curl &#8211; Client URL<\/h2>\n<pre>curl -s -S https:\/\/api.zenoss.io\/v1\/data-receiver\/models \\\r\n  -H \"content-type: application\/json\" \\\r\n  -H \"zenoss-api-key: $(cat ~\/.zenoss.key)\" \\\r\n  -X POST \\\r\n  -d @simple-model.json<\/pre>\n<ul>\n<li>-s : Silent, no progress meter or error messages<\/li>\n<li>-S : Show error, use with -s<\/li>\n<li>URL : target URL<\/li>\n<li>-H : Header.\u00a0 Can be used multiple times\n<ul>\n<li><code>-H \"content-type: application\/json\"<\/code><\/li>\n<li><code>-H \"zenoss-api-key: $(cat ~\/.zenoss.key)\"<\/code><\/li>\n<\/ul>\n<\/li>\n<li>-X : Request method (verb)<\/li>\n<li><code>-X POST<\/code><\/li>\n<li><code>-X GET<\/code><\/li>\n<li>-d : Data\n<ul>\n<li><code>'{ \"here\": \"is some content\" }'<\/code><\/li>\n<li><code>@use_this_file<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>JQ &#8211; JSON Query<\/h2>\n<ul>\n<li>Not required but provides pretty printing<\/li>\n<li>Not installed by default<\/li>\n<li>Good for redirecting CURL outputs\n<ul>\n<li><code>curl .... | jq<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>HTTPie Utility<\/h2>\n<ul>\n<li>Easy version of curl using Python<\/li>\n<li>Install via PIP<\/li>\n<li>Additional headers are just key:value pairs<\/li>\n<li>-b hides return headers and only displays the body of the message.<\/li>\n<li>Automatically assumes\n<ul>\n<li><code>content-type:application\\json<\/code><\/li>\n<li><code>-X POST<\/code><\/li>\n<\/ul>\n<\/li>\n<li>Automatically formats the output (jq not required)<\/li>\n<li>Example\n<ul>\n<li><code>https -b api.zenoss.io\/v1\/data-receiver\/models \"zenoss-api-key:$(cat key.file)\" &lt; data.json<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Shortcuts<\/strong><\/p>\n<ul>\n<li>Can use <code>:<\/code> for <code>localhost:<\/code><\/li>\n<\/ul>\n<h2>Scripting<\/h2>\n<p>1:46:00<\/p>\n<ul>\n<li>example.sh<\/li>\n<li>example_new.sh<\/li>\n<li><code>watch -n 30 .\/example.sh<\/code><\/li>\n<\/ul>\n<h1>Visualizing Streaming Data using Zenoss Cloud Dashboards<\/h1>\n<p>2:00:00<\/p>\n<h2>Overview<\/h2>\n<ul>\n<li>A Dashboard displays selected metrics of one or more entities over a specific period of time<\/li>\n<li>Dashboards are the primary mechanism for visualizing data sent to Zenoss Cloud through its data ingest API<\/li>\n<li>Another tool is the Zenoss Cloud Smart View feature, which will be discussed later.<\/li>\n<li>Collection Zones also have dashboards but they do not have access to streaming data.<\/li>\n<li>Dashboards are built from customizable tiles<\/li>\n<li>Zenoss provides a set of different tile types which allow you to choose the most appropriate visualization to display your data.<\/li>\n<\/ul>\n<h2>Scope, Entities and Metrics<\/h2>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-3579\" src=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics.png\" alt=\"\" width=\"1373\" height=\"645\" srcset=\"https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics.png 1373w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics-300x141.png 300w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics-1024x481.png 1024w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics-768x361.png 768w, https:\/\/wiki.thomasandsofia.com\/wp-content\/uploads\/2021\/07\/scopeentitiesmetrics-150x70.png 150w\" sizes=\"auto, (max-width: 1373px) 100vw, 1373px\" \/><\/a><\/p>\n<ul>\n<li>Information available to a dashboard is determined by the dashboard&#8217;s scope.<\/li>\n<li>The scope is a filter that limits the available data to a set of Entities.<\/li>\n<li>Each Entity may have one or more metrics associated with it.<\/li>\n<\/ul>\n<h2>Creating a Dashboard<\/h2>\n<ul>\n<li>Log into the cloud account used to create the API key earlier.<\/li>\n<li>Dashboards &gt; Create New<\/li>\n<li>Add Entities &gt; Search for the name used for the &#8216;source&#8217; field\n<ul>\n<li>This defaults to the username in the example scripts.<\/li>\n<\/ul>\n<\/li>\n<li>Click the `\u2713` &gt; Next<\/li>\n<li>Provide a new name &gt; Save<\/li>\n<\/ul>\n<h2>Configuring a Dashboard.<\/h2>\n<ul>\n<li>Enable Auto Refresh<\/li>\n<li>Select <strong>1H<\/strong> to display results received over the past hour.<\/li>\n<\/ul>\n<h2>Adding a Graph Tile<\/h2>\n<ul>\n<li>Click the pencil icon to edit the dashboard<\/li>\n<li>Select the graph tile from the Available Tiles box<\/li>\n<li>Change the Tile Title to &#8220;Random Number&#8221;<\/li>\n<li>In the Entity and metric search box:\n<ul>\n<li>Enter the &#8220;name&#8221; of the entity you created\u00a0 and select.\n<ul>\n<li>This did not work for a single entity.\u00a0 It only let me select the metric.<\/li>\n<\/ul>\n<\/li>\n<li>Enter <code>rand<\/code> and select random.number as your metric<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h1>Sending Data Using Python<\/h1>\n<p>2:19:00<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Resources Streaming Data Policy Day 1: s3.us-east-2.amazonaws.com\/s3.thomasandsofia.com\/zenoss\/training\/streaming-data\/20210303-Streaming_Data_to_Zenoss_Cloud-D1.mp4 Day 2: s3.us-east-2.amazonaws.com\/s3.thomasandsofia.com\/zenoss\/training\/streaming-data\/20210304-Streaming_Data_to_Zenoss_Cloud-D2.mp4 Day 1 Why Stream Data Short lived applications or services (docker containers) Container based services Microservices Serverless functions Need to Push data from application or service vs. having them Pulled Data that is not exactly &#8220;Metrics&#8221; but want displayed on a dashboard Business or Costs ..<\/p>\n<div class=\"clear-fix\"><\/div>\n<p><a href=\"https:\/\/wiki.thomasandsofia.com\/?p=3556\" title=\"read more...\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[64,59],"tags":[],"class_list":["post-3556","post","type-post","status-publish","format-standard","hentry","category-streaming-data","category-zenoss"],"_links":{"self":[{"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=\/wp\/v2\/posts\/3556","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3556"}],"version-history":[{"count":19,"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=\/wp\/v2\/posts\/3556\/revisions"}],"predecessor-version":[{"id":3809,"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=\/wp\/v2\/posts\/3556\/revisions\/3809"}],"wp:attachment":[{"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3556"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3556"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wiki.thomasandsofia.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3556"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}