About /robots.txt

In a nutshell

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:

User-agent: *
Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

There are two important considerations when using /robots.txt:

• robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
• the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.

Adding the .global.prod.fastly.net extension to your domain (for example, www.example.com.global.prod.fastly.net) via the browser or in a cURL command can be used to test how your production site will perform using Fastly's services.

To prevent Google from accidentally crawling this test URL, we provide an internal robots.txt file that instructs Google's webcrawlers to ignore all pages for all hostnames that end in .prod.fastly.net.

This internal robots.txt file cannot be customized via the Fastly web interface until after you have set the CNAME DNS record for your domain to point to global.prod.fastly.net.

Digital marketing (also known as data-driven marketing) is an umbrella term for the marketing of products or services using digital technologies, mainly on the Internet, but also including mobile phones, display advertising, and any other digital medium.

Digital marketing's development since the 1990s and 2000s has changed the way brands and businesses utilize technology for marketing. As digital platforms are increasingly incorporated into marketing plans and everyday life, and as people use digital devices instead of visiting physical shops,digital marketing campaigns are becoming more prevalent and efficient.

Digital marketing techniques such as search engine optimization (SEO), search engine marketing (SEM), content marketing, influencer marketing, content automation, campaign marketing, data-driven marketing and e-commerce marketing, social media marketing, social media optimization, e-mail direct marketing, display advertising, e–books, and optical disks and games are becoming more common in our advancing technology. In fact, digital marketing now extends to non-Internet channels that provide digital media, such as mobile phones (SMS and MMS), callback, and on-hold mobile ring tones

1. Do you see those attractive Web banner ads, Video campaigns, Mobile apps, Informative blog posts quite often while surfing the net? All these are nothing, but different forms of Digital Marketing!

As the world is getting more and more digitized in every respect, Digital Marketing is a booming career option today. With striking features like cost-effectiveness, instant response, flexibility, convenience, effectiveness, Digital Marketing is making a strong impact in the world of Marketing and Advertising. Especially, the start-ups prefer going digital to market their brand, products and services. Today ‘Digital Marketing’ and ‘Marketing’ both are becoming synonymous.

According to a Study made by Direct Marketing Association, Digital Marketing Industry is worth $62 billion! Also, according to eMarketer, Last year, advertising via mobile phones and tablets rose 180 percent, to $4 billion. This market is estimated to touch $7.2 billion soon. This rise is leading for high demand for professionals skilled in Digital Marketing.

1. SEO:Search Engine Optimization is an organic (unpaid) method of improving the ranking of web pages and their visibility on various search engines such as Google, Yahoo, Bing, etc. Optimizing webpages is a very vital aspect of Digital Marketing.

2. SEM:Another method of increasing the visibility of webpages is through sponsored placements and advertising. SEM specialists should know how to run PPC campaigns, how to purchase traffic through paid search listing to maximize visibility of web pages on search engines.

3. SMM:Social Media Marketing is gaining a lot of popularity with social media sites like Facebook, Twitter, LinkedIn, Pinterest and others. A good digital marketer should know how to harness the power of social media to market the brand image of the company.

4. Email Marketing:As it is important to engage your customers and leads with your brand, many organizations send periodic newsletters, email campaigns, autoresponders to its subscribers in order to keep them updated with new products/services added or anything worth sharing.

5. Mobile Marketing:More and more people are going mobile. Smartphones have become very trendy among all age groups. Mobile marketing provides instant updates, promo offers and information of customers’ interests. A person needs to be a mobile freak to get into mobile marketing.

6. Web Analytics:
Web Analytics is a very interesting aspect of digital marketing which involves traffic analysis, business and market research and enhancing the website traffic. For instance, Google Analytics gives a great insight into how your website is working.

Tips to make a career in Digital Marketing:

If you are keen in getting into Digital marketing, either as a fresher or if you want to switch to digital marketing, here how you can start:

1. Read a lot on Digital Marketing:
Just search for digital marketing or blog posts on digital marketing on Google, I am sure you will come across several useful blog post and articles explaining what digital marketing is all about. Through these, you will learn the basics, the strategies, techniques and various aspects of digital marketing. Read blog posts from Hubspot, Seomoz, Web Marketing Academy, Google, Unbounce, and so on to understand the world of marketing you are getting into. These blogs are very rich in information and content.

2. Attend Webinars/Events related to Digital Marketing, whenever possible:
Another way to learn more about Digital Marketing is by attending conferences, webinars and all those events which are related to Digital Marketing. Quite often major cities have meet-ups, where digital experts share their knowledge with people who are new to this arena. At such events, you also get the opportunity to meet like-minded people.

3. Choose your Specialization:
Though a general knowledge of all the areas of Digital Marketing is useful, once you have decided to get into Digital Marketing, you need to specialize. Are you passionate about writing articles or doing data analysis? OR Are you interested in SEO or Web Designing? Getting into a specific area of Digital Marketing will help you in getting more job offers than just having a basic knowledge. Companies are looking for experts in SEO, SEM, Email marketing, Web analysis and so on. And gradually, you can add other stuff to your knowledge bank!

4. Get Set Go!
Once you have decided your career path in Digital Marketing, just start working towards it. You can begin by writing a blog post, or creating a website or a web banner ad or improve your social presence, or doing traffic analysis. Initial few months, you will face technical issues in dealing with these aspects, however gradually, you will master the concepts and gain more confidence. Whether you are a blogger, an SEO Specialist or a Web Analyst, what you need is a lot of passion and creativity, as marketing itself requires constant innovation and ‘out of the box’ thinking. Working smart, keeping yourself updated with latest information, excellent communication skills, are key to success.

5. Focus On Business!Whatever marketing strategy you may follow, at the end of the day it should be business-oriented and should take the business a step ahead. All you technical knowledge is of no use if it is not driving the business. So, as a digital marketer, try to acquire the business acumen needed to implement the best marketing strategy.

With a lot of data analysis happening throughout the world, we have entered the era of Big Data! Everything is getting digitized, so why not marketing? Therefore, getting into Digital Marketing is a very lucrative career option today.

What are Dofollow Links in SEO?

Dofollow links enable to search engines such as Google to follow these links and reach out to your website. They provide websites with a link juice and a backlink. In case you get a webmaster who links back to you with such a link, then the search engine, as well as visitors, will be following you. The best way for gain a dofollow link is to allow a keyword in the anchor.

The example of Dofollow Link:

<a href=”http://www.google.com/”>Google</a>

Note: By default all the hyperlinks are dofollow. So, you don’t need to do anything to make a link do-follow.

What are Nofollow Links in SEO?

Nofollow links on the other links, on the other hand, do not let the bots of search engines to follow a link. This means that in case the website who is linking back to you has nofollow traits, it won’t pass the link juice. Only individuals can follow these links. Recently, Google clarified that it won’t consider nofollow link attributes. The value of these links is therefore quite less. Though it is wise to utilize Nofollow link attribute to the visitors who link, you won’t be able to get the link juice from them.

The example of Nofollow Link:

<a href=”http://www.google.com/” rel=”nofollow”>Google</a>

Important Notice:-

Now, Google has said that they will still count the nofollow link as an outgoing link in terms of the distribution of page rank from your page.

Types Of No-Follow

• Robots Meta Tag : <meta name=”robots” content=”nofollow” />

This tells bots/crawlers/spiders not to follow links on the full page.

• Link Attribute : <a href=”http://www.google.com” rel=”nofollow”>

This tells search engines not to count the link in terms of ranking pages.

Two Different Types of Backlinks in SEO

1. Good Backlinks

Google and similar search engines utilize numerous factors for deciding the quality of the backlinks for deciding the ranking of websites. Though they rarely share their exact mix, experts are of the opinion that majority of factors can be divided into four different categories-trust, relevancy, authority and diversity.

All major search engines want to offer their users with the search results which have the best results.

If the quality of these search results they offer declines. So also does the audience of the search audience. Therefore, it is quite logical to think about relevance as a factor for evaluating the quality of the link. A backlink is of more value in case it comes from a significant source. For example, if you have a site related to interior décor, an incoming link from a furniture website will be far more useful than the one from where it comes, like suppose a healthcare related website. However, search engines take into consideration a number of factors to decide relevance.

2. Bad Backlinks

A bad backlink, on the other hand, comes from a website which is not very trustworthy. After the Penguin algorithm in 2012, Google has become very strict against bad links. Before Google launched this algorithm, websites blatantly violated Google Webmaster Guidelines using black hat SEO. Basically, people were buying links in order to get better rankings and authority. Therefore, Penguin was launched in order to stop this malpractice. Instead of enhancing their rankings, this update penalized webpages which were connected with such backlinks.

A reliable search engine doesn’t try to return to pages which match the most suitable input query. It rather tries to answer the basic question. Search engines such as Google use a complicated algorithm to decide what results should return. Here are some of the latest Google updates which have brought about a change in the way of doing SEO: –

What is Google Panda Update?

The Google Panda Update is a search filter which was introduced with the main aim to stop sites with inferior quality of content to acquire top rankings on Google. Since its launch in 2011, Panda has been updated periodically. When an update occurs, sites which have been affected may survive, provided they have made the necessary changes. Panda can also catch sites that have escaped before.

The latest version launched in 2016 decreases or adjusts the ranking of a website rather than devaluing or ignoring the spam as in the case of Penguin 4.0. It is now a part of Google’s core algorithm. There will now be more updates to Panda. But this won’t bring any positive change in the condition of websites which had been previously impacted by earlier Panda updates.

What is Google Penguin Update?

Google Penguin Update was launched in April 2012 to catch which are believed to spam Google’s search results. They did this either by buying links or acquiring them through link networks designed mainly to enhance rankings on Google.  The newest update of Penguin, Penguin 4.0 was launched last year. As a result of this change in algorithm, Penguin has become more page specific. Now, Penguin undervalues spam by adjusting rankings on the basis of spam rankings instead of affecting a site’s ranking. Google has also confirmed that it won’t confirm Penguin updates in future.

What is Google Hummingbird Update?

Google Hummingbird Update was launched in 2013 to return better results. It pays attention to all words in a query. It takes into consideration the whole sentence rather than specific words. The main aim of Hummingbird is to ensure that the pages which match meaning perform better instead of the pages which match only a few words. Hummingbird update isn’t a penalty-based update like Panda and Penguin. Rather it is a change in the way Google shows results for different types of queries. This enables the search engine to understand the exact meaning behind a query instead of the separate terms in it. It is also better for dealing with conversational queries since the number of voice searches and mobile search users is on the rise.

What is Google Pigeon Update?

The Google Pigeon Update was launched in 2014 to offer more helpful, relevant and precise local search results which are closely linked with conventional web search ranking signals. It also enhances their distance and location ranking parameters. As an algorithm, Pigeon affects search results within Google Maps as well as Google Web Search. Earlier, people used to get very different results while searching Google and Google Maps. This has changed since the launch of Pigeon. The algorithm connects map search and web search in a more unified manner.

Summary

Google Algorithm updates have become something quite common these days. The launch of new updates like Panda, Penguin, Hummingbird and Pigeon have brought about a profound change in the way of doing SEO.