While working on the Deep Learning capstone “Deploying a Sentiment Analysis Model” project, most of my time was spent troubleshooting two common errors. They were ResourceLimitExceeded error and a CORS error of CORS policy: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

Resource Limit Error

The error I got was “ResourceLimitExceeded: An error occurred (ResourceLimitExceeded) when calling the CreateTrainingJob operation: The account-level service limit ‘ml.p2.xlarge for training job usage”

Fix

You need to apply for a separate instance for training and deployment. This Udacity resource suggests it does not matter whether you choose SageMaker Training or SageMaker Deployment

It DOES matter. If you are doing the Udacity exercise for training (as below) you need to request the SageMaker Training Resource Type and not SageMaker Hosting.

The training and hosting(deployment) instances have to be requested separately even if it’s the same instance, for example, ml.m4.xlarge, you need to request twice and specify training and hosting.

My own error occurred because I had requested SageMaker Hosting and was trying to use it for training. After contacting AWS and consulting online resources I was able to resolve it.

CORS error

The error I got was AWS sagemaker error has been blocked by CORS policy: No ‘Access-Control-Allow-Origin’ header is present on the requested resource. However the error is NOT because we need to change a CORS setting

Potential Fixes and hints:

1. For Step 7 (again): Use the model for the web app complete all the steps at one go, as many of the steps that follow interdependent

2. The endpoint needs to be deployed and running, if it is not running or not deployed, it will not work

3. The method used to execute the Lambda function is the endpoint created using API Gateway. This endpoint will be a URL that listens for data to be sent to it. So if the URL provided is not correct, an error is generated. Make sure you have the right endpoint in the lambda code. This was what was causing my error.

Additional Resources

• Is this movie a thriller? Exposing a SageMaker Deep Learning Model in an end to end serverless web application

• Resource limit when deploying

As a part of this scholarship I got the chance to work on open source group projects. I also developed an app that reflected my own history (as a former immigrant, and now a proud US citizen) and was a small attempt towards helping aspiring immigrants.

There are 100 civics questions on the naturalization test. During the naturalization interview, applicants are asked up to 10 questions from the list of 100 questions in English. You must answer correctly six (6) of the 10 questions to pass the civics test in English. This can be very hard for many immigrants. Some may know the answers but just getting up to speed on English. Others may want an interface that is more helpful than just the booklet.

So I used some of the code I had been developing earlier (you can see an early version here under citizenship test and completely revamped it.

The questions are shown, but not the answers, at least not until you click “show answer” . Some questions are dependent on location and for those you need to select a state. For instance the screenshot here shows the Senator for the state of California.

You can check out the live demo here and the github repo here under the development branch. And here is the wiki. The UI is inspired by the wonderful design work of Alexandra Hoefinger.

With all the learning from my PWA course, the scores were almost a perfect 100% on Lighthouse! Still trying to figure out the last bit to get to 100% on PWA.

Here are my main noob takeaways:

• Although indexedDB is complicated, it is really worth the effort and Jake Archibald’s IndexedDB Promised library makes it easier

• Caching of static assets improves performance dramatically

• Rather than develop stand alone functions for indexedDB it is far better to encapsulate all the functions within a variable

• Write methods to get and set data (in my case I don’t have update and delete but you may want to use those too). See link in Github. This makes it much easier to do all the work you need to do with your data.

• Check to see if an object store exists before doing writing data (see code for details)

• If you are making API calls, within the fetch statement, access the method you need from your idb variable and do stuff with your data

I hope this write up including the code for the IndexedDB code and the Cache API code is helpful for someone else! If you need any help regarding use of IndexedDB, drop me a line but don’t forget to clap first :-)

Default function parameters

Before we start: You know what a function and a parameter is

What it does The default function parameter allows you to set a default value for your parameter(s)

Why is it useful ? Not all parameters are created equal. Sometimes if a parameter value is missing the code can crash badly. default allows the programmers a nice safety net.

An example: Say you are writing a function that will perform a division by the value of one of its parameters. To avoid a division by zero you will have to do some detailed error checking.

function divide_old(a, b) {
  /* We have to make sure we have the right number of arguments and their values are not 0. We are simplifying here and assuming we are dealing only with positive numbers
*/
  if (arguments.length===2 && arguments[0]>0 && arguments[1]>0){
    return arguments[0]/arguments[1]
  }
  else if(arguments.length===1 && arguments[0]>0){
    return arguments[0]/1
  }
}

With the default value for the parameters in ES6 you can now write code as follows

function divide(a, b = 1) {
  return a / b;
}

This makes life a whole lot easier for both the user/caller of a function, who does not have to worry and of course for the function author.

For more see the MDN page

Fetch

Before we start: you need to know what is a call to a REST API means and what JSON data is, and what JSON data processing is.

What it does: Provides programmers with a really simple way to make calls to a REST API

Why is it useful: REST APIs give you access to databases with rich information that your program can use. Before ES6 the most common ways of doing this was to use a third party library, with the most popular being jQuery’s ajax.

An example: The example shows the mechanics of a fetch call and getting some JSON data. If you have done things the old way I am sure you can immediately appreciate the benefits.

function fetch_simple (url) {
  fetch(url)
  .then (response => {return response.json()}) // JSON parsing done for you
  .then(data => {
      // do whatever you want to do with the data
  })
  .catch (err => {
    console.log("===we have an error===")
    console.log(err.stack)
  })
}

For more see the MDN page

Sets

Before we start: You need to know arrays, and objects

What it does: It’s an iterable collection of unique elements. However this is not about what it is, but rather what it can help you do.

Why it is useful: You can very quickly remove duplicates from existing arrays.

An example (and more):

To remove duplicates just create a set from an array and then convert that set back to an array.

const nums_with_duplicates = [1,5,7,8,5,8,23,45];

a_set = new Set(nums_with_duplicates);

const nums_unique = [a_set]// the [] changes the set back to an array

While both objects and the new ES6 Map also provide the support for unique elements, Set preserves the iteration order of its elements. So when we need a collection where the order of iteration and uniqueness matter, Set comes upon top.

The pattern [...new Set (array_with_duplicates)] will always give you an array with the duplicates removed. Nice, right?

Basic operations with a set

The code to walk a set is also nice and compact

for (const month of months) {
    console.log(month)
}

For more see the MDN page

Rest parameter and Spread operator

Before we start: you need to know what arrays and parameters are. Also an iterable object

What it does: If you have been reading about ES6 and looking at code written in it I am sure you have seen the curious ... three dots popping up everywhere. These three dots have two names and two very different meaning in ES6. The naming and the meaning depends on what section of the code they appear:

• ... is called a rest parameter when used in function declaration, for example, in function foo (a, ...b){//do stuff}, the dots make b a rest parameter. The dots also mean that the second parameter is an array with an unspecified number of elements.

• Any other place you see those dots they are spread operators. The spread operators most commonly appear in array literals like this: [...a, 4, 5,6] Here also the dots represent an array of unspecified number of elements.

The a must be an iterable object like arrays, strings and the ES6's new Maps and Sets.

Why is it useful: When used as rest parameters, the dots “pack” many variables in to an array. So in the function function foo (a, ...b){//do stuff} if we call foo with foo(5,6,9), this would mean that a is 5 and b, because of the three dots is an array[6,9]

When used as a spread operator the dots “unpack” variables. See the following function:

function bar (a, ...b){ 
  console.log(bar); 

/* [6,9,10] Note it only starts from the second number because the first parameter is not part of the spread operator */

  console.log(...b);// 6 9 10 //notice it unpacks [6,9,10] 

 }
bar(5,6,9,10)

The first three dots are rest parameters and “pack” the variables into an array. The second set of three dots “unpack” the array into the individual elements. This makes it easier to work with arrays in functions.

An example

function foo (a, ...rest) {
  console.log("\n\nvalue of the first param is  " + a)
  console.log("the rest of the params are in an array: ")
  console.log(rest)
  console.log("unpack them to a list as follows:")
  console.log(...rest) 
  return "Done"
}

foo (5,6,9,11) // a will be assigned 5 and the rest will be [6,9,11]
foo (5,6,9,11, 2) // a will be assigned 5 and the rest will be [6,9,11, 12]
foo (5) // a will be assigned 5 and the rest will be empty
foo (5,6,[9,11]) // will be assigned 5 and the rest will be [6,[9,11]]

Copy and paste the above to the console to see how it works

Feel free to dig deep into many interesting usecases for rest parameters and spread operators.

But as a newbie I find the following two patterns very useful:

The spread operator make array concatenation much more natural.

joined_array = [...first_array, ...second_array, ...third_array]

You can combine it with ES6 Sets to remove duplicate from arrays

const unique = [...new Set(non_unique)];

For more see the MDN page and this MDN page too

Destructuring

Before we start: You need to know sets, arrays, parameters

What it does: Destructuring allows a programmer to break a compound structure like an array or an object down to its elements (or properties in case of objects) and assign those “destructured” pieces to variables. [a,b] = [1,2] is now a valid Javascript statement, which assigns 1 to a, and 2 to b.

Why it is useful: It makes code simpler and short. See example for more

An example (and more):

Get rid of the pesky temp variable and three lines of code and swap variables in one shot.

var a = 1;
var b = 3;
[a, b] = [b, a]; // 
console.log(a); // 3
console.log(b); // 1

Parse function returns like a Pro

function f() {
  return [1, 2];
}

var a, b; 
[a, b] = f(); 
console.log(a); // 1
console.log(b); // 2

Remove duplicates from Arrays

Create a duplicate free arrays in one shot. This one needs to know Sets too!

const unique = [...new Set(array_with_duplicates)];

1. Create a set from the array_with_duplicates array, this will remove the duplicates

2. Use the spread syntax to unpack the set elements

3. “destructure” the unpacked element back into an array

For more see the MDN page

And finally putting everything together..

Javascript takes on the big guns

Many languages (such as Perl) are famous for allowing programmers to express fairly complicated commands in just one line. Javascript was never known for this. With ES6 it is slowly getting there.

Consider the following array that contains the month names when the temperature dipped below a threshold.

const low_temp_months = ['January','January', 'February', 'March','November', 'December','December'];

As you notice there are some duplicates perhaps caused by multiple readings submitted for the same month. If you want the month names to be unique you can do the following:

1. Create a set from the low_temp_months array, this will remove the duplicates

2. Use the spread syntax to unpack the set elements

3. “Destructure” the unpacked element back into an array

All the three aspect can be implemented in a very nice one liner!

const winter_months = [...new Set(low_temp_months)];

When I applied to the Mobile Web Specialist track I was not sure what I would get out of Udacity. I considered myself a middling self-taught web programmer. (Here’s a blog that documents my learning journey and here’s the links (1 and 2) for my most popular app with over half million users). If I could learn anything I wanted to on my own, what would be Udacity’s value proposition for me ? So my expectations were fairly low even when I heard I got the scholarship. Fast forward three months.

I am blown away by the experience.

Why ? Two words — community and open source.

I quickly became part of a rich evolving learning community.

Four specific experiences were invaluable:

• For the first time in my life I worked on not just one but two open source projects, one of which I initiated both built around maps. One was the Navi project with a very large team and a great lead Bryan who was very helpful. Another one was Mappa an all women project, one that I initiated just with Khusbu but quickly grew to include some very talented and hardworking women. Mappa started off with some of the code I used for a Udacity course using knockout js. Halfway through the project we switched to React, another interesting experience of changing track midway and managing the implications on the project timeline. On Navi, I focused on making documentation easy to read and understand. In the process I became familiar with git, github, learned to read other people’s code, realized the importance of documentation and much more.

• Working with students from different programming backgrounds taught me a lot. There were absolute beginners, those with some experience, and others that with significant experience in open source and programming. With such a diversity of experience, collaboration is the only way the community moves together as a whole. The more experienced ones helped the less experienced ones, and the less experienced ones helped the more experienced by asking questions that need to be answered.

• A community needs tools to grow and to communicate. Udacity chose Slack and Forums. What even Udacity could not predict was how the tools would be used. The organic growth and innovative use of the tools provided (Slack channel, forums) and tools not provided (Google Hangouts, opened issues on Github) was fascinating and could probably be a PhD in itself. Some folks started a list of useful resources, I wrote a tool to capture video captions from Udacity lessons and many others organized meetups and study sessions.

• I was exposed to the advantages and disadvantages of the democratic experience where coders of different ability often have the same voice, and work together in a transparent atmosphere. You can join any channel you want, most ask for help when they need it without shame, and people are empowered to work on projects even if they don’t have much experience.

This led me to immediately contrast this way of working to the typical workplace that I am so used to — hierarchical, top down management structure, and with many black boxes.

Working on open source made me think about and reaffirm the importance of transparency, collaboration, balancing top-down design with bottom-up innovations in code and in life

Given the diversity of programming backgrounds, and seeing the difficulty in communication between those who knew a lot and those that were beginners, I have been inspired to tackle the gap: to really understand a topic in depth and create accessible materials that explain the topic to beginners or those with relatively less experience. This was the single biggest impact of the Udacity/Grow with Google experience on my life.

While most Web API provides multi page results to a call, most API documentation will only talk about how to get the first page of those results.

It is almost impossible to find guides to pagination!

Zendesk gets a B

Here is an example from Zendesk:

If at all they have instructions on pagination, it will be hidden away somewhere in the API documentation

Here’s the documentation on pagination on Zendesk. This one was hard to find but not impossible.

Google gets an F

Sometimes the instructions may be hidden way deeper than necessary. See below for a search on “pagination” on http://developers.google.com.

I am assuming the documentation people leave the details out because for most professional developers this is trivial.

Not so for a newbie. If you are a code n00b, you are have struggled with the stubborn OAuth, pored over API docs to get the perfect call that will get you the data for your app, and finally late in the evening, you are getting the data back! The last thing you want is to cross yet another hurdle.

So here’s what I have: There are two common ways API designers tell you that there are more pages of results.

One is via a next page field in the result, which will contain the url for the next page. If there are no next pages the field will not be present in the result. Zendesk uses this approach.

The other way is to return the token for the next page, which you append to your next call. Most Google APIs uses this approach. Twitter uses a similar next cursor key.

In either case the solution boils down to the following

1. Have an array to store all the results

2. Make a call and get the result and push them to the array

3. Enter a loop if the next page field or page token is present

4. Make the new call either using the next page URL or the token

5. Get the result

6. Optionally — give the server a few milliseconds break. Be nice.

7. Here’s what I did for Zendesk. I used a do-while loop, but while-do works just fine of course.

function zd_get (cmd) {
  var url = ZD_BASE+cmd;
  var results =[];
  var service = getService();

  if (service.hasAccess()) {
    do {
      try {
        var response = UrlFetchApp.fetch(url, {headers: {'Authorization': 'Bearer ' + service.getAccessToken(),}});
      }
      catch (e) {
        return e;

      }
      var result = JSON.parse(response.getContentText());
      results.push(result);
      url = result['next_page'];
      Utilities.sleep(250); // give the server a break

    }  while (result['next_page'] !=null);
    return results;
  }
  else return null;

}

1. Here’s what I did for a Google API call

function zd_get (cmd) {
  var url = ZD_BASE+cmd;
  var results =[];
  var service = getService();

  if (service.hasAccess()) {
    do {
      try {
        var response = UrlFetchApp.fetch(url, {headers: {'Authorization': 'Bearer ' + service.getAccessToken(),}});
      }
      catch (e) {
        return e;

      }
      var result = JSON.parse(response.getContentText());
      results.push(result);
      url = result['next_page'];
      Utilities.sleep(250); // give the server a break

    }  while (result['next_page'] !=null);
    return results;
  }
  else return null;

}

Prerequisites

• You are familiar with basic Node (i.e. you know loops, if-thens etc.) and understand the basic callback pattern.

For example, you are good to go if you have done (or can do) the Node challenges in Quincy Larson’s Free Code Camp

• You are NOT an experienced programmer. You will be bored :-)

• You have an account on Cloud9 and you have created a workspace on Cloud9.

Terms

1. Web Server: The Web is vast and complex but its core functionality is strikingly simple. The Web is built around two classes of programs: servers and requesters. Servers run continuously (unless they are “down”) and listen for requests for web pages. When it receives a request it provides a copy of that page (if it has access to it) to the requester.

2. HTTP protocol: The communication convention these two classes of program use to talk to each other is the famous http protocol.

3. Requesters, browsers and a web page: The requesters are programs that can ask for a specific web page from a server. A requester that can display the web page received from a server is what we know now as browsers. All browsers are requesters but not all requesters are browsers. For example, when you write some code to access and process JSON data using web API, your code is a requester and not a browser.

4. Express: Express is a program that makes setting up your own web server very easy. It is a written in Node. Anything you can do in Express you can also do it in pure Node but it is a lot of work. Besides TJ Holowaychuk and Node.js Foundation contributors, have already solved the problem for us.

Setting up express, creating an app and running it in Cloud9

1. Install express in a directory of your choosing (see link for details)

$ npm install express --save

1. Create a new javascript file

1. Use the code below

var express = require('express');
var app = express();

app.get('/', function (req, res) {
  res.send("<h1>hello world</h1>");
});

app.listen(8080, function () {
  console.log('Example app listening on port 8080!');
  //call this app from https://<workspace name>-<user name>.c9users.io
});

1. One challenge I had was that I did not know where the app was running. The typical *http://localhost:<port number>/ *as referenced in various express resources is not going to work since the app is running on Cloud9 servers and the command above works only to test apps running locally.

On Cloud9 it will be http://<workspace name>-<user name>. If you’ve forgotten what these are, you can check in your browser window. For example in the picture here, the user name is tanyagupta (me) and the workspace name is test. Therefore in this case you would go to http://test-tanyagupta.c9users.io.

You can also use the preview button to do so.

NOTE: The app “runs” only while you are logged in.

If you wanted to get into the details of the code, here is the same code, heavily marked up with comments:

// Standard set up for the express code to work with your code
var express = require('express');
var app = express();

/*

Routes a get request, and in this simple case - to the server's root. It sends a 
call back to handle the request. Not much will happen here, the call back is hardcoded  
to send a hello world string. But in theory you can do something with the req variable 
in the "/" directory, create a dynamic html and send that back
*/

app.get('/', function (req, res) {
  res.send("<h1>hello world</h1>");
});

/*

Firing up your server. For most of us noobs - all that matters here is
a) remember the port number - 8080 (in this example). If you use some other port 
number make sure you mention it explicitly when you make a call to the server.
b) the console statement is just a note to tell you the server is running

 */
app.listen(8080, function () {
  console.log('Example app listening on port 8080!');
  //call this app from https://<workspace name>-<user name>.c9users.io
});

JSON

This post is about how I had been processing JSON data from Web API calls, why the approach I chose was “poor taste” code, and what I did to fix it.

Prerequisites

• You are familiar with basic Javascript (i.e. you know loops, if-thens etc.).

  For example, you are good to go if you have done (or can do) the Basic JavaScript challenges in Quincy Larson’s Free Code Camp (link)

• You have cursory knowledge of web APIs. (i.e. you know what they are used for). You know a bit about relational database and SQL.

  This does not mean you have actually used them in your code (although that would be nice).

• You are NOT an experienced programmer. You will be bored :-)

Preamble (a.k.a. why this post?)

The inspiration for this post comes from Brian Barto’s thought provoking and popular (2.1 k recommends and counting) article on “good taste” coding. Brian used an example given by Linus Torvalds (you know that guy who single handedly wrote the kernel of one of the world’s most robust, efficient and elegant operating system and gave it away for free) in an interview to drive home some software development best practices. While I encourage everyone to read the article, my post here is about what I ended up doing after reading his article.

I am a self-taught newbie hobbyist programmer. And my partner (in both code and in life) who is trained in “legacy” CS, says I am bit of a “code brat.” I code first, think (of algorithm and stuff) later. I code for my pleasure (i.e. not for money, for now). Once I get an app working and tested, I deploy it and never look back (well I do listen to user complains and fix them, but that is about it). Brian’s article nudged me to take a critical look at the code I have written over the last two years (as I learned Google Apps Script and built Google Doc/sheet Add-ons and web apps using Google App script).

It is humbling to find examples of “poor taste” programming in one’s own code. But it is fun to fix them and talking about them is almost therapeutic. In this post I will describe one such fix (the fancy word for it is refactoring).

On processing the (data) web using APIs and JSON

Web APIs give you access to databases with rich information that your program can use. Here’s an introduction with detailed example using the New York Times Web API. Most modern Web APIs return data in JSON format. JSON is a subset of Javascript and parsing a string to a JSON (i.e. a Javascript) object is trivial.

var data = JSON.parse(string_result_of_api_call)

Boom! The Javascript object datanow contains all the information that was sent back in response to the API call.

Things get a bit messy from here.

The “mess” explained

The use of Web API to get data from a web service is conceptually very similar to making a query into a database. For example, a SQL query select * from cars where price <10K(get me all the cars priced <10K), will return a table with rows that match the query criteria.

In JSON the results would look a bit different.

[{make:'Toyota', model:'Camry', year:1999, price:7K, doors:'2DR'},{make:'Nissan', model:'Sentra', year:1988, price:3K, doors:'2DR'},{make:'Jeep', model:'Cherokee', year:1988, price:7K, doors:'4DR'}{make:'Jeep', model:'Wrangler', year:1984, price:9K, doors:'2DR'}]

1. The result is a valid Javascript object (an array in this example)

2. Each row of the result is represented as an element of an array and the fields name are given for every value.

Although the look is different, the underlying structure is still not that different.

Now let’s bring the Web’s messiness. What if some of the fields are absent from some of the rows. This usually does not happen in professionally built and curated in-house databases but irregularities like this is the norm when it comes to the Web and results from the Web API calls are no different.

If the results were returned in an SQL style rows they would look like the following (where we are missing information about the Nissan’s year and its drive train type the model for one of the two Jeep).

Processing this data is fairly straight forward. We loop through the data, for each row, we check if a value is null (or blank) and do what we need to do with the data that is available. Our code does not break, just because there is a null or an empty value.

In JSON the data would be something like

[{make:'Toyota', model:'Camry', year:1999, price:7K, doors:'2DR'}, {make:'Nissan', model:'Sentra', price:3K},{make:'Jeep', year:1988, price:7K, doors:'4DR'}, {make:'Jeep', model:'Wrangler', year:1984, price:9K, doors:'2DR'}]

For Nissan, as expected there is no key for year and doors. For one of the Jeeps the model is missing.

My n00b inclination was that the processing of this data is still quite simple. I will need to add an additional check to see if a field is present before accessing its value. I will access the key and see if accessing the key returns undefined. If it does not, then I capture the value. In our Nissan example, trying to access the key year will return undefined. So my pattern was

if (key != undefined) {do_some_thing(object[key]}

It worked well until I encountered complex (i.e. real life) JSONs.

JSON is a nested tree like structure, rather than a table structure. While the keys of a JSON object has to be a string or a number, the values can be objects. The values of the keys of these child objects can themselves again be objects. Many real world JSONs can be 8 or 9 level deep. Using my pattern this would mean that I would have a 8 or 9 level nested if-thens.

Try debugging that.

“Poor taste” a detailed example

We continue with the car domain. Let’s say we are tasked to print the values of certain fields from the inventory. We will display the value of the fields if they are available. If the fields are not there or if the value of a field is not present we will not display anything (i.e. write a blank).

//Note that this is an INEFFICIENT way to solve the problem
function noob_way() {

  // get test data from a test data utility
  var cars=get_data_set("basic");  


  for (var i in cars){
    if(cars[i]["model"]!==undefined) { 
      Logger.log(cars[i]["model"]); // Logger.log() is Google App Script's console.log  
    } 

    if(cars[i]["trim"]!==undefined) {
      Logger.log(cars[i]["trim"]); // Logger.log() is Google App Script's console.log  
    }

    if(cars[i]["year"]!==undefined) {
      Logger.log(cars[i]["year"]); // Logger.log() is Google App Script's console.log  
    }

    if(cars[i]["wheel"]!==undefined) {
      Logger.log(cars[i]["wheel"]);
      if(cars[i]["wheel"]["wheel_size"]!==undefined){
        Logger.log(cars[i]["wheel"]["wheel_size"]); // Logger.log() is Google App Script's console.log  
      }
      if(cars[i]["wheel"]["wheel_metal"]!==undefined){
        Logger.log(cars[i]["wheel"]["wheel_metal"]); // Logger.log() is Google App Script's console.log  
      }
    }
  }
}

Checking for model, trim and year is easy because the keys are at the first level. But wheel_size and wheel_metal are nested fields. We need to first check if the parent field (in this case wheel) exists before we access them. You can see how the complexity increases in line 21-29. Again imagine processing fields that are 8 or 9 level deep.

Before I describe my refactoring let’s look at an interesting feature that is available in many programming languages.

Short-circuiting — booleans

As logical expressions are evaluated left to right, they are tested for possible “short-circuit” opportunities using the following rules:

1. false && (*anything)* is short-circuit evaluated to false

2. true || (*anything)* is short-circuit evaluated to true

Here’s a useful partial truth table

A partial truth table

Taking advantage of rule # 1 if you write cars["wheel"] && cars["wheel"]["wheel_size"] AND if cars["wheel"] does not exist (i.e. it is false in Javascript) the rest never gets evaluated.

Basically the short-circuiting can convert a nested if else error checking for undefined fields to a simple single line of code. The depth of a field will not increase the complexity it will just add more clauses with &&s.

Finally, here’s the cleaned up code

//Smarter way to access the data
function noob_smart_way(){

  var cars=get_data_set("basic");  
  var format_string = "%s %s %s %s %s";

  for (var i in cars){
    Logger.log(format_string, cars[i]["model"] || "",cars[i]["trim"] || "",cars[i]["year"] ||  "",
                cars[i]["wheel"] && cars[i]["wheel"]["wheel_size"]    ||  "",
               cars[i]["wheel"] && cars[i]["wheel"]["wheel_metal"] || "") 
    // Logger.log() is Google App Script's console.log  
  }   
}

TL;DR: Use short circuit of && to check for keys in a deep JSON instead of a nested if-else. This will significantly reduce your code complexity. Thread together all the keys from the top level until you reach the key whose value you want with bunch &&s.

k_0:{k_1: {k_2:{k3: … kn}} should become k_0 && k_0[k_1] && k_0[k_1][k_2]&& … &&k_0[k_1][k_2]….[kn]

In the last two years I have spent 2300 hours in coding and have written 23,342 lines of code. In that code I have made 349 real world JSON calls followed by nested if-then-else. Say it took me on the average 1 hour to code/debug each of them. That is still 350 hours or 15% of my coding time.

This pattern literally takes a minute or less to write and it is mostly bug proof. That is 350 minutes or about 6 hours.Go figure!

Over time I introduced enhancements such as enabling downloadable word clouds but did not know if those enhancements were indeed being used. I googled the essentials, set up Analytics tracking, and included the basic tracking code in my script file.

This basic tracking code is well explained on the Google Developers analytics.js page

I quickly realized that the basic set up did not result in useful information on usage for Add-Ons as the counts were vastly exaggerated (tracking was done using cookies that were not persistent and resulted in information per use, and not per user). (The number of installs displayed in the developer dashboard was not much help for analytics). Then I found an Analytics resource that presented the code to be used and a blog with a discussion of the cookie issue by Romain Vialard:

This (ultimately) with a lot of trial and error, finally worked.

However it took me a very long time to understand what this code was doing and how to use it. Not a fun experience. So in case there is somebody out there looking for a step by step explanation of why the above works and what it means, I posted my code on gist with a LOT of comments. I hope it helps someone to save some time and get to tracking Add-On usage rather than googling!

Leave a message if you have any additional tips or comments

/* 
 The most common way to track users in Google Analytics is by using cookies. But sometimes 
 there are platform and security limitations that will not allow cookies to be set or 
 stored for multiple sessions. The following code uses the localStorage property (available 
 now in most browsers) to provide an alternate way to the cookie-based solution. The code 
 is heavily commented to help even the most casual users understand the overall workings 
 of the code. Feel free to ignore this part if you know what you are doing. 

This work is based on a Google example: 
https://developers.google.com/analytics/devguides/collection/analyticsjs/cookies-user-id 
I have simplified it and added comments

There is a blog about this code in medium:link and some basic concepts explained in the 
comments section: 
*/

// First I make sure that there is localStorage available in the browser. The code 
//only works if localStorage is available.
if (window.localStorage) {

  /* The first step to setting up a google analytics tracker is to issue a GA create 
  command. Typically it is done as follows: 
  ga ('create', 'UA-XXXXXXXX-5', 'auto')

  This tells the GA code to create a tracker for the property with an id 'UA-XXXXXXXX-5 
  and tells GA to use a default domain to store the analytics cookie.

  But I want to create the tracker a bit differently. I want to do three things:

  1) I want GA not to use cookies
  2) I want GA to use a client id to identify the browser so that I can track the user 
  and not every session of the user
  3) I want to store the client id locally (ie on the browser) so that it can be passed 
  on to GA for the next time 

  */


  ga('create', 'UA-XXXXXXXX-5', // these two parameters remains exactly the same as 
     //Google's standard tracking code
     // I pass the added information via the following object:
     {
       'storage': 'none', // Tells GA not to store cookies as local storage is being used
       'clientId': localStorage.getItem('local_client_id') // Tells GA to set a clientId 
   //to identify this browser/user

       /* Gotcha alert */

       /*        
       This tripped me up. So warning to you too: The value of clientId is initially null! 
       This is because there is nothing in localStorage yet and therefore 
       getItem ('local_client_id') will return null. GA when creating the tracker will 
       generate a random unique number as a client id      
       */


     });


 /* 
  Once the tracker is created I want to get the random client id generated by GA 
  and store it locally. 
  I want to do this because next time when the tracker is created the locally 
  stored client id will be passed, so that the GA knows it is NOT a new user 
  (because GA already has the client id)

  To do this Google offers a cool way to get tracker information. I call ga and 
  pass in an anonymous function with a single parameter. When ga gets a call 
  with a function (with a single parameter) 
  it puts the created tracker data in that parameter.

  The function passed earlier requests the client id from ga and stores 
  in the localStorage  

 */


  ga(function(tracker_data) { 
    localStorage.setItem('local_client_id', tracker_data.get('clientId')); 
   // get the client id from GA and store it locally
  });

}
else {/* fallback if there is no local storage - In this case, GA will 
count each and every hit even when the same user accesses the app multiple times*/
  ga('create', 'UA-XXXXXXXX-5', 'auto');
}
// Send the hit - it will only be counted when the client id matches  
ga('send', 'pageview');

GA concepts:

Tracker objects: A JS object created by GA that can collect and store data and then send that data to Google Analytics.

Tracking id: A string like UA-000000–01. It is included in tracking code to tell Google Analytics which account and property to send data to.

Client id: The client id is a unique, randomly generated string that usually gets stored in the browsers cookies, so subsequent visits are associated with the same user.

The weirdness of Javascript is hard enough to deal with if you are an experienced programmer. It is much harder for a new programmer.

My first encounter with Javascript’s strangeness was with the in operator. The in operator is a nifty tool to check if a specified property is present in a specified object. For example we can check if a certain key is present in a object that has several key value pairs.

var mycar = {make: "Honda", model: "Accord", year: 1998};
"make" in mycar  // returns true

"mileage" in mycar  // returns false

Unfortunately the in does not work in an intuitive (and useful) way when used with arrays.

In the example below, I would really like it if Javascript’s in would work like this:

var trees = ["redwood", "bay", "cedar", "oak", "maple"];

"bay" in trees    // returns false

The reason it does not work (and it should not) is the particular way Javascript’s array objects are implemented. They are internally viewed as a key value pairs of index and the corresponding element. For example the trees above is stored internally as something like:

{0:"redwood", 1:"bay", 2:"cedar", 3:"oak", 4:"maple"}

And yes, 1 in trees will return true, but that is not useful to me as a programmer.

I am relatively new to programming (here’s my story in medium) and I do not code professionally. But even I encounter several scenarios almost every day where I need to check if some item is in an array or not. Naturally this weirdness in Javascript puzzled me a bit. From what I found out, this is available in other languages (Python, Ruby) so — why not Javascript? I googled a bit and found this on how to implement a “linear search” in Javascript. Which I did.

To check if an item is in an array, I have to iterate through the array elements until the item is found in the array.

function inArray (item, theArray) {
  var found = false;
  var i=0;
  while (!found && i in theArray) {
    found =(item === theArray[i]);
    i++;
  }
  return found;

}

But it was not satisfying. I could not tell then what exactly about it dissatisfied me — at that point it was just something I did not think was cool. I now know better. Please read on.

1) Array.indexOf()

The Array.indexOf() method returns the first index at which a given item can be found in the array. If it is not present it returns -1. Thanks to Nikolay Digaev for pointing me to this (another is Array.includes() — which unfortunately is not currently supported by Google Apps Script where I mostly work on these days. The Array.includes() method returns true or false based on whether an array includes a certain element.)

Here is a simple use case for Array.indexOf():

function item_in_array(){
 var fruits = ["Bob", "Rob", "Amanda", "Cheryl","Susan"];


  Logger.log(fruits.indexOf("Susan")) //returns 4 which is the index of Susan
  Logger.log(fruits.indexOf("Frank")) //returns -1 since the item is not found

}

2) Convert the array to an object literal and use the “in” operator

I figured out a way to use the in operator in my use cases. I started converting my arrays to objects. I would store each (unique) item as a key and just put a blank in the corresponding value.

The Hack: convert the array to a typical object with items as keys and empty values

**Creating the object:**

  var hack_object_literal = {};
  for (var i in test_data) {
    hack_object_literal[test_data[i]]="";

  }

Careful readers may notice that my hack ignores duplicate items when building the object. Which is fine since all we want to know if an item is present. Not how many of them are present.

Now I can use the in operator to check if the item is present in the object which in effect tells me if the item is present in my array.

function inArrayMyHack (item, arrayDataAsObject) { 
  return item in arrayDataAsObject; 
}

Recommendation: use Array.indexOf() — easier andshorter

Courtesy: Pixabay

An experiment to determine speed

Computer science purists believe that data structures should be used for the tasks they are originally designed to do, as otherwise, it results in unmaintainable code. One may deviate from this principle only if there is a compelling reason to do so. Leaving the values empty in a collection of key-value pairs seem to be a major deviation. So I decided to see if there is a compelling reason for my “hack.” And I found it.

I implemented the three functions: the linear search function as inArray, Array.indexOf() and inArrayMyHack

1. For every size of arrays that I created:

a) I sorted the array element and created a sorted version of the array

b) For every element in the sorted array I called both the functions giving them the element and the array. So for a 200 size array both the functions will be called 200 times each.

I recorded the time it took for them to complete.

Note for inArrayMyHack I also had to convert the array to an object before I made the call. I added that time to the performance time of inArrayMyHack — but there is a catch to that to0, read on..

Try { there are significant efficiency gains}

The graph and the table below shows that either of the two methods are way faster than the linear search.

The hack and Array.indexOf() are close initially but the Array.indexOf() methods slows down once size increases. I talked to some coder friends who told me that the reason for this performance is the in operation uses a very fast algorithmic technique called hashing. This technique produces a performance that is (almost) constant time regardless of the data size. Linear search time increases as the data size increase. The data conversion time is very negligible compared to the speed gained by hashing.

Speed test

Catch with the hack{We bomb if the arrays are not known at coding time}

There can be situations where I do not know about the arrays at coding time. In such case I will not be able to convert the arrays to objects before making the call. Things will slow down if I have to do a lot of searches each time with a different unknown array.

Bottom line: Use Array.indexOf() — will fit for most use cases unless you are dealing with large sizes and you have some knowledge of the arrays at coding time.

I started my journey of code. Here’s the TL;DR

• 2000 hours of weekends and evenings — check

• Close to 60,000 users for one of my software products — check

• 5 software products in Chromestore — check

• Can code up an app in a week, something that would have taken me 3–4 months, just two years ago — check

And most importantly a lifelong desire to continue as a (hobbyist) programmer.

The longer version:

I am not an expert programmer. Novice++ at best. My code is very green. I know it. My partner Abir Qasem, who has a Phd in Computer Science, agrees. But I have no fear of programming anymore and I do have a few products that people like and use. I am deeply humbled by the knowledge that I have so much more to learn. But at the same time I am very happy and proud of whatever little I have achieved.

I kept records of my progress mostly to learn from my experience and to know if and when I need to throw in the towel. But since this journey has at least, on a personal level, become a success story, I want to share. I have inspired by the journey of others. So I am sharing my story in the hope others may find it useful. And I like infographics too. So ‘nuff text.

(thanks to Free Code Camp, Quincy Larson, Luca, Ben at Google, and Google Plus Group for Add-On Developers)