Correlation And Causation By Example

src: https://github.com/DataScienceWorks/correlation_causation_blogpost/blob/master/cc_blog_post.ipynb

What you should know quickly?

Given the observations of two features A and B: we observe correlation between features A and B, when we see a pattern where A and B change its values at the same time. When the values of A and B, increase or decrease together, we say they are positively correlated. When the value of A increases if we find that the value of B decreases proportionately and vice-versa, we say they are negatively correlated.

Correlation is what we can visually identify by plotting the values of features in the graph and compare their trends for patterns. With this we cannot say what is causing what. In other words, you cannot claim one feature causing the other.

When the change in one feature results in the change in the other, we call it causation. When we find correlated features, we dig around the domain, do more research and homework to increase our domain knowledge to claim that one feature is causing the other.

Using PySpark 2 to read CSV having HTML source code

When you have a CSV file that has one of its fields as HTML Web-page source code, it becomes a real pain to read it, and much more so with PySpark when used in Jupyter Notebook.

Advice from a Patient to a Doctor

What a Doctor should understand?

The source of this photograph is unknown. But I guess this is the best piece of advice that a Doctor can get and one that every Doctor should read at least once everyday until this becomes their second nature in practice.

10 secrets to becoming an Engineering Leader — quickly!

1. Wear a blazer and a torn jean. You can boost it further by wearing spectacles. All this with a big fat belly gives that desired first impression.

2. Suffix the certifications next to your name. Don’t be shy about it. You have after all, p̶u̶r̶c̶h̶a̶s̶e̶d̶ earned it.

Quick Lessons in Data Science practice

1. Basic statistics and mathematics is your compass in the field of Data Science. You must have it to have a sense of direction.
2. You are better-off choosing one of R or Python or Matlab or any other platform, and become a master in it than attempting to get dirty with all of these.
3. Right from the start, version-control your work with a tool like Git/Github/Bitbucket. It can save you from a lot of headaches.
4. Don’t under-estimate the importance of understanding the problem domain. Domain knowledge is the secret weapon for Data-Manipulation.
5. Don’t under-estimate data cleansing. It pays to cleanse your data.

50 Common Misconceptions In The World Of Software Development

We all have our share of mistakes. We evolve over time learning our lessons. This post is to help you introspect your thoughts on varied things that float in the realm of software development. Below is a compilation of "some" common misconceptions that we hear and get influenced by for bad.

What makes a bad CTO?

Submit a form with Enter key in React

Imagine you  have a simple form like the one shown in the below screenshot:

Book Review - Dear Boy: Lord Chesterfield's Letters to His Son

Read this book and you will fall in love being a conservative.

Read this book and you will understand what it means to care as a parent for their child.

Read this book and you will appreciate the values behind philosophy.

Read  this book by all means for it gives you a chance to become not just literate but educated as well.

My grandpa gifted and read this book to me to kindle my  interest reading this book  again and again during  my childhood. This book sure made a dent in my life for good. I hope and wish it does the same to you as well.

Go get it at Amazon - now!

All about WAF for AWS Exam and later

Web Application Firewall (WAF) in General

• WAF filters, monitors, and blocks HTTP traffic to and from a web application.
• WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers.
• By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security mis-configurations. A WAF can be considered a reverse proxy.
• WAFs may come in the form of an appliance, server plugin, or filter, and may be customized to an application. The effort to perform this customization can be significant and needs to be maintained as the application is modified.

All about ENI for AWS Exam and later

• An ENI (Elastic Network Interface) is a logical networking component in a VPC that represents a virtual network interface card (NIC).
• If ENI, think virtual NIC.
• An ENI can include the following attributes: 
• a primary private IPv4 address 
• one or more secondary private IPv4 addresses 
• one Elastic IP address per private IPv4 address 
• one public IPv4 address, which can be auto-assigned to the network interface for eth0 when you launch an instance 
• one or more IPv6 addresses 
• one or more security groups 
• a MAC address 
• a source/destination check flag 
• a description
• You can create an ENI, attach it to an instance, detach it from an instance, and attach it to another instance in the same subnet.
• Multiple Elastic IP addresses can be applied to an ENI. 
• Multiple IP addresses can be assigned to an ENI. 
• An ENI has a dynamically assigned private address in the assigned subnet, and can optionally have a dynamically assigned public IP address as well. 
• You can create and configure ENIs in your account and attach them to instances in your VPC. 
• An ENI's attributes follow it as it is attached or detached from an instance and reattached to another instance. 
• When you move an ENI from one instance to another, network traffic is redirected to the new instance. 
• You can also modify the attributes of your ENI, including changing its security groups and managing its IP addresses. 
• Each instance in your VPC has a default network interface, called the primary network interface (eth0) that is assigned a private IPv4 address from the IPv4 address range of your VPC. You cannot detach this primary network interface from an instance. 
• You can create and attach an additional network interface (i.e ENI here) to any instance in your VPC. The number of network interfaces you can attach varies by instance type.
• Attaching multiple network interfaces to an instance is useful when you want to: 
• Create a management network. 
• Use network and security appliances in your VPC. 
• Create dual-homed instances with workloads/roles on distinct subnets. 
• Create a low-budget, high-availability solution.

AWS CSAA Exam Tips

The Exam Preparation

• First and foremost. The journey is more important than the result. When you cherish your journey, you can rise up to any situation. If not for this, you are purely relying on luck that only makes you fragile. Grow your strength. Do your round of preparation to digest the concepts. 
• Begin right with note-taking. Prepare your own notes. Trust me, it helps! You can compare it later with others to fill gaps if any. This is in many a sense going beyond the exam preparation and helps facing the real world situations like job-interviews, solution-consulting, problem-solving at work. Ask how? Because it helps you internalize the concepts.
• Practice more and more mock tests. Don't skip it, ever. It pays to tone your mind for the exam nitty-gritties. Knowledge is one thing, warming-up your mind to be alert for exam-tricks are yet another thing. Also, mock-tests help plug your gaps in the grey-areas.
• Specifically, look out for those mock tests that helps you familiarize with the test's UI. Familiarity is part of the preparation to boost your confidence. 
• Do take mock-tests but be sure if the answers provided by them are right. There are many mock-tests that come with wrong answers. So, after every test, do your homework for the correctness of your answers.
• Drink lots of water.
• Eat good deal of fruits (not the stupid nutrient tablets).
• Eat less or just enough to stay fit. Don't over eat, it only sickens you.
• Topics Check-List to know what is in/out for CSAA pre-Feb-2018 exam: 
• IAM - Identity Access Management
• KMS - Key Management Service
• WAF - Web Application Firewall
• VPC - Virtual Private Cloud
• Route 53
• Internet-Gateway
• Egress-only Internet Gateway
• Subnet
• Route Table
• NACL - Network ACL
• Security Group
• VPC Flow Logs
• Bastion Hosts
• ENI - Elastic Network Interface
• Elastic IP
• IP Addressing - CIDR (Classless Inter-Domain Routing)
• NAT Gateway
• NAT Instance
• VPN Connection
• Direct Connect
• VPC Peering
• VPC Endpoints
• VPC Endpoint Services
• EC2 - Elastic Cloud Compute
• AMI - Amazon Machine Image
• Placement Group
• EC2 Auto-Scaling
• AWS Lambda
• CloudWatch
• Cloud Formation
• S3
• EFS
• EBS
• Storage Gateway
• Snowball
• CloudFront
• API Gateway
• ELB (mostly Classic, and sometimes Application load balancer). Network Load Balancer is not yet included as yet AFAIK. Good to know about it though ;)
• RDS - Relational DB Services
• DynamoDB
• Redshift
• Elasticache
• Kinesis
• SQS - Simple Queue Service
• SNS - Simple Notification Service
• SES - Simple Email Service
• SWF - Simple WorkFlow service
• Elastic Transcoder
• AWS Marketplace conceptually
• AWS CLI conceptually
• AWS SDK conceptually
• ECS - Elastic Container Service
• Elastic Beanstalk
• For each of the above topics in the check-list read the FAQs, and User Guide for better understanding.
• Pick any of the online course for the exam out there in the market. It helps filling the gaps in your understanding. Now which online-course should you pick? It's purely your personal preference.
• There are some topics that form the bed-rock of cloud operations like VPC and its components, the EC2, ELB, NAT Gateway, RDS. etc. You should get dirty with these topics hands-on and you should never ever skip that if you want your learning to stick. Go do it on the AWS Cloud to have an appreciation of how much you can do in AWS cloud and how well Amazon has done that to help you take control if required. Don't ever miss this. If you don't have experience, take a course that runs you through the steps is a definitive help.

Experience Report - AWS Certified Solution Architect Associate Exam

I cleared CSAA (Pre 2018 release / old pattern) exam​​ with 91%.

I don't give much importance to certifications and more so to percentages. In spite of that there is always this tizzy when one takes an exam. I both love and hate this at the same time :-/