Scientific Computing: Big Data Analytics

What is Big Data?

Big Data is collections of set of large and complex data. It is not just about collecting them, it also about storing and warehousing this data. Warehousing the data include functions like capture, storage, search, sharing, transfer, analysis and visualization.

Traditional database system:

In the history we have come access various types of database systems like the navigational database system, relational database system, SQL database and NoSQL database systems. All these database system have a common problem which is the limitation in storage. In this digital world we are generating huge amount of digital data in our day-to-day life knowingly or unknowingly. Therefore the limitation is storage has risen as a critical problem.

Why Big Data Analytics?

Big data solves this problem by accommodating all the data we generate. Big data supports lots of data-types. Both structured and unstructured data can be processed and handled in Big data.

• Discovery: As it stores various types of data, it is difficult to get a solution by querying it. Therefore we need some automated mechanism to search the data for us.
• Iteration: With huge set of data, it hard to find where to explore the data to get out results. So iterative approach is used.
• Mining and Predicting: Mining data and predicting results have become serious business. There are lots of start-up companies using this as the core idea. For example, The Climate Corporation is a San Francisco based start-up company which provides insurance to the farmers based on huge records of climate and weather data.
• Decision Management: Concluding or deciding a thing from very huge set of data using traditional database is practically impossible. Therefore Big data plays key role in any decision making task. 

Computer Graphics: Autodesk's 3ds Max & Maya

3DS MAX:

It is the best 3D architecture modeling software. It is formerly known as 3D Studio Max. It is used by lot of game developers, animation movie makers and commercial advertisement makers.

Features:

It has all the features that make the architecture of any body we need create. It has lots of plug-ins which can be added depending upon the animators need. Some of the important features are MAXScript, Character Studio, Skinning and Texture assignment. MAXScript helps is creating scripts that automate the repetitive motions of the animation. Character Studio helps to animate virtual characters. Skinning is used to create the precise and exact skeletal formation. Texture assignment is used to design and provide texture to the animated body.

Modeling techniques:

• Polygon modeling makes it possible to create polygons. Every polygon is created individually and all the characteristics are added to it.
• NURBS: Non-uniform rational B-spline eliminates the rough edges in the polygon model. It makes the surface of the polygon smooth.
• Surface tool: It is used to provide any type of surface to the polygon which is created

MAYA:

Autodesk’s Maya is the best software to create character models and animation. It is nowadays used in all the animated movies.

Components:

There are lots of components in Maya which provide different functionalities. Some of the important components are classic cloth, nHair, fur and fluid effects.

Classic cloth has all the cloth patterns which we use in real-world. nHair is used to create all the lengthy human hair. Fur is used to create hair on body which is usually used to created animal s hair. Fluid effects we can be used to create non-elastic fluid like smoke, fire, explosion and so on.

These software are proprietary software. There are lots of tutorials in YouTube to learn about them. Here is a link to learn 3ds Max and Maya.

Communications and Security: Email security

Email has become inevitable mode of communication in our day-to-day life. Even though there are many new technologies like chat, messenger and video chats are in the market, email still has its value. Email is preferred for both formal and informal communications. Email has its own weakness and it is prone to threats. Let us see some of the techniques we can use to have secure communication via email.

Create a Unique Password: Using same password for all the email accounts is not advisable. If one account is attacked eventually all the other accounts having same password will be attacked. In the worst case, we must have a unique password at least for the master account which has all you potential details.

Encrypt your data: Data sent through email must be encrypted. Double encryption can be done to protect sensitive data. This will help to prevent the loss of data.

Use digital certificates: Sign all your sites using digital certificates. Good practice is to save your digital certificates in routers or load balancers rather than web servers.

Spam filter on email servers: Use any spam filter to avoid spam. SpamAssassin is an Apache software which prevents unwanted email from entering your inbox.

Scan for Viruses & Malware: If something in your mail looks suspicious, run the malware and virus scanner. Virus does not affect your inbox every time but still it is safe to safe to run virus scanner every time. 

Artificial Intelligence: Google Brain

Google Brain is an artificial intelligent machine which speaks to the user and answers the questions. It is a deep learning research project at Google. Andrew Ng, the Director of the Stanford Artificial Intelligence Lab founded the Google Brain project at Google, which developed very large scale artificial neural networks using Google's distributed compute infrastructure.

Google Brain is a fascinating project which uses set of algorithms from machine learning. It also uses artificial neural networks. Researchers try to train the Google Brain uses various sets of training data so that it can be used to enable high quality speech recognition, email spam blocking etc. Even the Google’s self-driving car concept is a part of the Google Brain project.

To teach the machine to distinguish between cars and motorcycles, the researchers need to collect tens of thousands of pictures that have already been labeled as ‘car’ or ‘motorcycle’ to train them. People working on this project say that they are create a neural network which simulates a new-born brain and introduce it to YouTube videos for a week. They inferred that it started to learn unlabeled images from the YouTube stills. This small-scale simulation of Google Brain learned what the word ‘cat’ is from the YouTube’s unlabeled stills. This is known as self-taught learning.

They are working on implementing this on all possible areas like speech recognition and email spam blocking. 

History of Computer Science: Who invented computer?

There is no easy answer for the question who invented the computer. Computer was not invented by one person, but many. It was not invented in a single break through but a series of incremental steps. It continues even today as new generations.

First automatic computing engine concept

Charles Babbage is known as the “father of computer”. He created the first Analytical Engine which contained the Arithmetic Logic Unit(ALU). It also had integrated memory and it is the first general purpose computing concept.

First concepts modern computer

Alan Turing proposed a concept of Turing machine to print symbols in 1936. It is considered as the first modern computer. The Turing machine has a tape of symbols which computes the logic of any computer algorithm.

First digital computer

ENIAC is the first electronic general-purpose computer. It has individual panels which perform different functions. The ENIAC was invented by J. Presper Eckert and John Mauchly at the University of Pennsylvania and began construction in 1943.

First commercial computer

In 1942, Konrad Zuse begin working on the Z4, which later became the first commercial computer after being sold to Eduard Stiefel a mathematician of the Swiss Federal Institute of Technology Zurich on July 12, 1950.

The first PC (IBM compatible) computer

The first personal computer was introduced by IBM in 1981. The computer is referred to as the Acorn and had an 8088 processor, 16 KB of memory, which was expandable to 256 and utilizing MS-DOS.

File Sharing: Peer-to-peer

Peer-to-peer file sharing is different from traditional file downloading. In peer-to-peer sharing, you use a software program (rather than your Web browser) to locate computers that have the file you want. Because these are ordinary computers like yours, as opposed to servers, they are called peers.

The process works like this:

• You run peer-to-peer file-sharing software on your computer and send out a request for the file you want to download.
• To locate the file, the software queries other computers that are connected to the Internet and running the file-sharing software.
• When the software finds a computer that has the file you want on its hard drive, the download begins.
• Others using the file-sharing software can obtain files they want from your computer's hard drive.

The file-transfer load is distributed between the computers exchanging files, but file searches and transfers from your computer to others can cause bottlenecks. Some people download files and immediately disconnect without allowing others to obtain files from their system, which is called leeching. This limits the number of computers the software can search for the requested file.

Advantages:

• Fault tolerance
• Performance
• Cost efficiency

Disadvantages:

• Release of personal information, bundled spyware, and viruses downloaded from the network.

Some of the well known peer-to-peer file sharing programs are Bit Torrent, Shareaza, Limewire, eMule etc. Visit this site for more information regarding peer-to-peer file sharing programs.

Data Structures: Trees

A data structure is an arrangement of data in a computer’s memory (or sometimes on a disk). Data structures include arrays, linked lists, stacks, binary trees, and hash tables, among others.

Tree is an acyclic connected graph where each node has zero or more children nodes and at most one parent node. 

A tree is a non-empty set one element of which is designated the root of the tree while the remaining elements are subtree of the root.

Properties of Tree data structure

Depth: It is the length of the path from the root to that node. It is counted by the number of edges traversed.

Height: It is the longest path from that node to its leaves. The height of a tree is the height of the root.

Leaf node: It has no children. Its only path is up to its parent.

Types of trees

Binary: Each node has zero, one, or two children. This assertion makes many tree operations simple and efficient.

Binary Search: A binary tree where any left child node has a value less than its parent node and any right child node has a value greater than or equal to that of its parent node.

Traversal

Three different methods of traversal are possible for binary trees. They are 'preorder', 'postorder', and 'in-order'. They differ from each other by the order in which they visit the current node, left subtree and right subtree. 

Preorder: Current node, left subtree, right subtree 

Postorder: Left subtree, right subtree, current node 

In-order: Left subtree, current node, right subtree 

Hacking: Hacker Classification

Computer hacking is modifying computer hardware or software to accomplish a goal outside of the creator’s original purpose. People who engage in computer hacking activities are often called hackers.

Malicious attacks on computer networks are officially known as cracking, while hacking truly applies only to activities having good intentions. Most non-technical people fail to make this distinction, however. It is extremely common to see the term "hack" misused and be applied to cracks as well.

Hacker Classification:

White Hat Hacker (Ethical Hacker): They are paid and working for a Government Agency or Companies to deliberately Hack into their Computer Mainframe to find any potential Weak Points in a Computer's Mainframe, and then figure out how to fix them.

Black Hat Hacker: They are also known as a cracker. They are the type of hacker that has malicious intent whenever he goes about breaking into computer security systems with the use of technology such as a network, phone system, or computer and without authorization. His malevolent purposes can range from all sorts cyber crimes such as piracy, identity theft, credit card fraud, and so. He may or may not utilize questionable tactics such as deploying worms and malicious sites to meet his ends.

Grey Hat Hacker: They are a combination of a black hat and a white hat hacker. A grey hat hacker may surf the internet and hack into a computer system for the sole purpose of notifying the administrator that their system has a security defect, for example. Then they may offer to correct the defect for a fee.

Elite Hacker: Elite hacker is the name utilized by the community with the aim of identifying those individuals who are deemed to be as experts in their line of work. These people are actually on the “cutting edge” of both the computer and network industry (Example:Master of Deception).

Script Kiddie: This is a derogatory term for black hat hackers who use borrowed programs to attack networks and deface websites in an attempt to make names for themselves.

Open Source: Misconceptions - Free vs. Open Source Software

Open source software is software whose source code is available for modification or enhancement by anyone. Source code is the code which normal users cannot see. Developers can modify this code to change or enhance the functionality of the software. Proprietary software are the other software in which we don’t have access to the source code.

The term 'free' in Free Open Source software refers to freedom, not monetary cost. Though most free open source software is indeed free in price, the term 'free' is referring to the freedom to use the software and source code.

Free vs. Open Source Software:

Free Software was introduced by Richard Stallman in early 1980’s. He started GNU project to avoid dilemma of either denying his neighbor a copy of software/source code, or breaking either copyright or Non-Disclosure Agreement. Free Software is focused on the freedom to share with your neighbor.

Open Source was later introduced by another community. Open Source is focused on effective development using the bazaar model (as described in The Cathedral And The Bazaar).

Free Software:

Free Software is software which users can freely:

• Run, for any purpose
• Study and modify for their own needs (requires source code)
• Redistribute copies (charging for them if they wish)
• Improve and release the changes to the public (requires source code)

Open Source Software:

The consumer of an open source program has the rights to do the following things to the source code:

• Read it
• Use it
• Modify it
• Distribute it
• Charge money for services related to it, such as copying or support, so long as they do not infringe on the freedoms of others

More details about Free Software and Open Source Software are available here.

Misconceptions:

1. Free means there is no monetary cost
2. Open Source software is low quality
3. If the software is free, then who is going to pay you to write it?
4. Open Source doesn't scale
5. Open source has no support
6. Open Source does not generate business revenue
7. Large-scale development isn't efficient

Agile: Scrum

Scrum is the best agile software development framework for managing software projects or application development. It based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. 

A key principle of Scrum is its recognition that during a project the customers can change their minds about what they want and that unpredictable challenges cannot be easily addressed in a traditional predictive or planned manner. Therefore we use Scrum which is a iterative agile software development framework.

Roles:

1.   Product Owner: Responsible for ensuring that the Scrum product backlog remains prioritized as more is learned about the system being built, its users, the team and so on.

2.   Development Teams: Responsible for building and demonstrate what they have built. Product Owner decides what to build next based on the demonstration.

3.  Scrum Master: It ensures that the Scrum process is used as intended. The Scrum Master is the enforcer of the rules of Scrum, often chairs key meetings, and challenges the team to improve.

Sprint:

It is the basic unit of Scum development. The duration is fixed in advance for each sprint. Each sprint is preceded by a planning meeting and followed by a review meeting.

Planning Meeting- Tasks for the sprint are identified and an estimated commitment for the sprint goal is made.

Review or Retrospective Meeting- The progress is reviewed and lessons for the next sprint are identified.

Advantages of using Scrum:

1.      Changes said by customers can be easily made.

2.      Bugs and errors can be found easily.

3.      Improved productivity.

4.      Predictable project completion and delivery.

LinkedIn: Professional Networking

LinkedIn is a professional networking site – so if you see Facebook as being about your social life, LinkedIn is the equivalent for work. Founded in December 2002 and launched on May 5, 2003, it is mainly used for professional networking. As of June 2013, LinkedIn reports more than 225 million acquired users in more than 200 countries and territories.

Why to use LinkedIn:

More than just a resume - Share your volunteer aspirations on your LinkedIn profile.

Keeping in touch – From my personal experience I know that not all employee have accounts in social networks like Facebook. But all employees have a LinkedIn as they feel it is very important. Therefore we can be in touch with employees even if we leave that company which in turn improves your networking.

Obtaining referrals – It’s easy to pass referrals to your network and of course there’s the opportunity to receive them too.

Great recruitment tool – Having strong presence on LinkedIn increases your chances of being hired as a freelancer or being offered a new job. Use LinkedIn Advanced Search to find potential candidates or advertise in job section.

Obtaining relevant news on your industry – Use LinkedInToday – it’s like a personalized newspaper and has just been updated – and LinkedIn Groups to find out gain valuable insights and to find out what’s going on in your sector.

Top 10 favorite features in LinkedIn:

Even though there are many features, the above mentioned ten features are considered as the favorite by the users.

QR Codes: QR codes vs Barcodes

The QR code system was invented in 1994 by Toyota's subsidiary, Denso Wave. QR codes were initially used by automotive industry in Japan. QR codes consist of four types of data (numeric, alphanumeric, binary, Kanji).

Why and how QR codes are useful:

You don’t have to own a business to use QR codes. They are free and anybody can own it. They are used to keep track of your things.  For example, they are used in libraries to track books, CDs and DVDs. Even personal information can be stored in QR codes.

However they are used to track and store information, they are mainly used by companies, banks and institutions for promoting them and their products.

QR codes vs. Barcodes:

The QR code and barcode are examples of information matrix. However a significant difference in the two is that while a barcode only holds information nicely in the horizontal direction, a QR can do so vertically as well. This is why QR codes are referred to as two-dimensional, because they carry information both vertically and horizontally. Another direct result to this is greater potential to carry information in a smaller space. Compared to a barcode, it’s no competition at all.

Therefore we conclude that QR codes are more advantageous than barcodes.

Social Networking and Security: In Social Media Marketing

Social media marketer must be careful when giving suggestions to their clients for promoting client’s brand. There are several online tools which we and our client can use to monitor how the brand is getting endorsed or liked by people.

Social networking sites are real boon to the entrepreneur because they can create pages for their brand and their brands can be advertised free of cost. Nowadays there is a special team called social media marketing team within the marketing team of every company to advertise their brand and their products in social networking sites like Facebook, twitter and YouTube. The main features in social networks that popularize the brand and their products are Like and Share.

The tools used to monitor the brand can be used by our clients to check the progress in promoting their product. Therefore we must make our promotions pleasing and appealing with offers provided by the client thereby increasing the consumer participation.  

On the other hand, social networks are risky as they are subjected to everyone’s comment. Anyone can register their thought about the brand as a comment. There are chances of negative promotion of the brand because of some bad comments. Of course, the comments which are not legitimate can be flagged.

Thus Social Networking can both promote and demote the brand. It all depends on the way how we share our thoughts on the social networking sites.

Welcome

Hi everyone!! Welcome to my blog..

I am a Computer Science guy pursuing Masters in Computer Science at San Jose State University. I have been working with Capgemini and Synopsys in the past. Currently I am working as a College Co-op at EMC². Based on my work experience, I have pretty decent knowledge on Software development, Web Application development and Big Data Analysis.

"Computer Science is no more about computers than astronomy is about telescopes."

-        Edsger Dijkstra

Computer Science is a very vast and interesting field of study. It is not just about computers, it is about computations, algorithms, data structures… to software development, robotics, big data. I have always been interested towards data. How they are organized, managed and maintained securely. It is really fascinating to know how huge volume of high-dimensional data is analyzed. I have good knowledge about SQL database. I have some idea of how the NoSQL works. I also have worked with BigData which is booming in the Silicon Valley. I learnt about Hadoop (BigData analytics tool) and did a real-time project which fetched me a job in this field.

Apart from that, I am also interested in Web development. You can notice that from this blog’s theme. I was not satisfied with the themes which Blogger provide. So I have my own custom theme for my blog ! 

As recently as the year 2000, only one-quarter of all the world’s stored information was digital. The rest was preserved on paper, film, and other analog media. But because the amount of digital data expands so quickly (doubling around every three years) that situation was swiftly inverted. Today, less than two percent of all stored information is non-digital. Therefore, there emerges a need for a new form of database technology called Big Data to deal with these massive digitally stored data.

I am quite fascinated by Big Data and I am learning the emerging technologies to make myself stable in this field. The beginning of knowledge is the discovery of something we do not understand.