Database Journals

Postgres vs. MongoDB for Storing JSON Data – Which Should You Choose?


https://www.sisense.com/blog/postgres-vs-mongodb-for-storing-json-data/




How to Decide if PostgreSQL is the Right Reporting Database for You


https://www.sisense.com/blog/decide-postgresql-reporting-right/




Pros and Cons of Using MySQL for Analytical Reporting


https://www.sisense.com/blog/pros-cons-using-mysql-analytical-reporting/




Database Corner: Beginner’s Guide to MySQL Storage Engines


https://www.sisense.com/blog/beginners-guide-to-mysql-storage-engines/


10 Useful Ways to Visualize Your Data


https://www.sisense.com/blog/10-useful-ways-visualize-data-examples/

Polygot Programming

 In the age of Polyglot programming ,found some interesting links to learn different polyglot programming languages

 To learn R & Numpy

To learn PostgreSQL

To learn GO

To learn Scala

To learn Node.js , Python & Ruby

Agile & DevOPS

Agile is defined by a set of values and principles enshrined in the Agile Manifesto. DevOps is Agile. On the other hand, Agile has become defined by a set of practices that are almost universal to Agile implementations - things such as standups, team rooms, test-driven development, a product owner, iterations, and so on. In that sense, DevOps change Agile

Agile project culture is very team focused.A DevOps pipeline looks like a waterfall process, consisting of requirements followed by implementation, followed by various kinds of system level testing, followed by deployment. What makes is not waterfall, however, is the fact that the pipeline operates continuously. That is, every portion of the pipeline has something in it - it is a non-stop flow.

Behavior-Driven Development Versus Test -Driven Development

The practice of Test-Driven Development (TDD) is deeply entrenched in Agile culture. Teams that practice it are often viewed as advanced, whereas teams that do not are considered by many in the Agile community to be less advanced.TDD has come under fire, and there are many people who feel that it is not the best approach for everyone in all cases. it turns out that DevOps does not need TDD for an effective pipeline. What DevOps needs is Behavior-Driven Development (BDD).

BDD consists of defining tests from a user perspective. Thus, tests are "end-to-end" because they are defined in terms of the behavior of the system - not the behavior of system components or even more granular "units". Historically it has been difficult to implement a BDD approach because one needs to assemble the entire system as an integrated unit to perform the tests. However, virtualization - the technology that has enabled cloud computing and DevOps - makes it possible - even easy - to create local instances of system components, so that one can assemble an integrated system very early. That makes BDD possible. Thus, one can use a test-first approach in which tests are defined at a behavior level, instead of a very granular unit level, as is the case with TDD.

Blockchain

Block chain works by creating a shared, distributed ledger of the transactions as they happen, building a permanent and immutable history of every transaction in the chain. Rather than going through a trusted third party such as a bank or stock market, transactions are validated and added to the chain by common consensus across the network. Any additions are validated in all copies of the ledger instantly. The chain is essentially under equal control by all its participants, with no one party overseeing contract execution. It is secured by a blend of asymmetric data encryption, and digital identities through which participants are authenticated.

Leadership in the age of Digital Complexity

Success in the Digital Age depends on courageous leadership, transformative thinking, and a zero-tolerance policy toward internal silos that stand in the way of speed and agility. Because business-as-usual will be out of business within five years.

There has never been a time - in recent memory - when there was more pressure on executives to come up with new approaches to corporate strategy. Why? Because we’re living in the age of rapid, relentless change, with increasing volatility – economically, socially, politically, technologically.

Nextgen language JULIA

Julia has 250,000 users, and has broad use cases in the domain of applied mathematics. Some of the current use cases of Julia include:

Self driving cars: UC Berkeley researchers use Julia to optimise model predictive control for the Berkeley Autonomous Race Car (BARC).

Astrophysics: Lawrence Berkeley Labs, Intel, and UC Berkeley researchers are using Julia and a NERSC supercomputer to increase the speed of astronomical image analysis by 225 times.

Risk models at Aviva.

Cancer Genomics: UK cancer researchers use Julia to model tumor growth, informing interpretations of cancer genomes.

Aviation: The Federal Aviation Administration is using Julia to develop the Next Generation Airborne Collision Avoidance System.

Asset Management: Conning is using Julia in large scale Monte Carlo simulations for insurance risk assessment solutions.

Julia is a very algorithm-oriented language, and it’s not about business transactions.

Technology Stack

GoJek’s tech stack is varied and they use Java, Ruby, Go, Clojure, JavaScript and PHP. The entire brain behind finding a driver and allocating to it the customer is on Clojure because a functional language is needed to solve that kind of problem. Ruby and Ruby on Rails power GoJek’s portals, and Java powers their backend (the company use gRPC instead of REST). Go (programming language)solves the concurrency problem and takes all high processing loads. They use MySQL for old parts and newer ones use PostgreSQL with MongoDB at few places.

GoJek has 120 microservices right now and the company processes 180 million internal API calls per second.

People don’t realize how complex it is to have these 16 products using the same user ID, same credentials, 16 different OMS(Operations Management Suite) sending you 16 different receipts but appearing the same. Because your history is coming from 16 places. We use a lot of message buses, and queues(event driven framework) to solve this problem.

Team created mircoservices (a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services) and attacked problems. For example, if you get a lot of login one day, then if you had login service you could just increase the instances for that service. That’s how they created the chain of commands which could do these things one at a time. Microservices helped a lot in scaling.

---------------------------------------------------------<<<<>>>>>>>>>-----------------------------------------

Analytic Tools

Machine Learning

Couchbase

HDFS

Algorithmic Pricing

One of the hardest questions for anyone selling a product is: “How much should I charge?”


Today’s Internet companies are charged with putting a price tag on products and services in a people marketplace, with minimal historical data or comparative models to lean on.


Leading companies explain the magic behind why their dynamic pricing models work.

Good data scientist

Qualities that make a good data scientist

A good data scientist = data hacker + programmer+ analyst+ coach+ story teller+ artist. 

1) You need to enjoy an iterative process of development 
2) You also need to have a good intuition for when to stop
3) You should be comfortable with failure
4) You should be driven by curiosity
5) You need to have a good data intuition
6) You need to have a good sense of metrics

Hadoop FrameWork Diagram