Regex for URL to sites
regex,sitemap,regex-negation,regex-greedy,regedit
Keep things simple: .*/f/$ vs .*/flight/$ ...
How to get Google to re-index a page after removing noindex metatag?
web-crawler,sitemap,meta-tags,google-webmaster-tools,noindex
Google usually fairly quickly crawls your pages. Inclusion into index is a bit slower, and getting reasonable search rank takes time. Look at your web server log to confirm that google bot did crawl your pages, you can search for exact page in google and it usually comes up, but...
MS CRM 2013 sitemap links to external pages broken when coming from a different sitemap link to an external page
dynamics-crm,sitemap,dynamics-crm-2013,webresource
This is an issue that was introduced in UR1 of Dynamics 2013. We follow an identical process with our product and we hit this same problem after upgrading. What is happening is that there is some JS code buried deep inside Microsofts Dyanamics JS, that is called before you navigate...
Change “Opportunities” button name in navigation pane using Site Map
dynamics-crm,sitemap,dynamics-crm-2015
The easiest way to do this is to change the display name of the opportunity entity. ...
Creating A PHP Sitemap XML Script
Your server probably has short tags enabled which causes the <? in the XML declaration to throw a parse error. The easiest work around is to just echo that line out. echo '<?xml version="1.0" encoding="UTF-8"?>' ?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> It also looks like you're missing the closing bracket to your while...
delete a node from sitemap xml
c#,sitemap,mvcsitemap,google-sitemap
If you're using C# you should use System.xml.linq (XDocument) You can remove a node like so: XDocument.Load(/*URI*/); var elements = document.Root.Elements().Where(e => e.Element("loc") != null && e.Element("loc").Value == "http://www.my.com/en/flight1"); foreach (var url in elements) { url.Remove(); } ...
Automatic sitemap.xml
Whether you create it manually or automatically, the result should be the same, and consumers would not now about how it got created in the first place (they can’t see behind the curtain). But in case of manually, if you only intend to update it monthly, you might loose 1...
Sitemap on dynamic generated content
indexing,seo,web-crawler,sitemap,dynamic-data
No one (neither human visitors nor search engine bots) knows if a document is created dynamically or if it exists as a static file. A search engine would have no reason to handle a link to http://example.com/show.php?ip=127.0.0.1 differently from a link to http://example.com/ip/127.0.0.1. By using URL rewriting (e.g., mod_rewrite for...
Sitemaps for country domains which are directories
You should gather all your sitemaps into a single .XML file, and then submit it to the Google Webmaster Tools. If you wish your sitemap to be indexed by other SE (w/o manual submitting), then name your XML sitemap as sitemap.xml and place it to the site's root, e.g.: site.com/sitemap.xml
Missing XML tag This required tag is missing. Please add it and resubmit
You're serving different sitemap content to Googlebot. This is what we (Google) get when we try to fetch your sitemap: <?xml version="1.0" encoding="UTF-8"?> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> </sitemapindex> Try to fetch the sitemap itself with Fetch as Google in Search Console (former Webmaster Tools) or with a tool like Testuri) ...
CakePHP sitemap error
I would recommend putting this in your Controller's action instead of setting the header value using PHP in your layout: $this->RequestHandler->respondAs('xml'); It's the real 'Cake' way of doing it, and should solve your problem (which is probably whitespace or something being output before the header). Of course, you need to...
PHP - Dynamically append web page to sitemap.xml
You would first want to parse what you already have, if you're adding on to it: $sitemap = simplexml_load_string(YOUR_ORIGINAL_XML_HERE_AS_A_STRING); After that, let's say you want to create a new url node. You can do this by using the addChild function on any SimpleXMLElement (this creates a child XML node, or...
Create SiteMap in ASP.NET MVC
asp.net-mvc,routing,sitemap,mvcsitemap
We had this same issue for MvcSiteMapProvider for MVC 4 and up. We solved it by removing and then replacing the UrlRoutingModule in the web.config file. <configuration> <system.webServer> <modules> <remove name="UrlRoutingModule-4.0" /> <add name="UrlRoutingModule-4.0" type="System.Web.Routing.UrlRoutingModule" /> </modules> </system.webServer> </configuration> ...
How/What to use to make a 3D sitemap
http://olafureliasson.net/uncertain An example of using 3 dimensional navigation. You are correct that Three.js/canvas can do this for you but wouldn't there be an easier solution: something like http://tympanus.net/Development/MorphingSearch/ that allows you to give the wow factor without having such a large learning curve...
Disallow google robot from robots.txt and list sitemap instead
Google index only with crawling... The best thing to do for you, is to disable the geolocation script when you detect a Google robot (or other) You can recognize them in various ways: HTTP_USER_AGENT or HTTP_FROM, or IP...
Robotstxt Google Searchresults
sitemap,google-search,robots.txt
You can use the Allow keyword to give access to a URL in a Disallowed directory. Allow: /nl/sitemap.xml Disallow: /nl/ ...
php fsockopen is NOT working
fsockopen accept only hostname. please use cURL library or simple file_get_contents function ...
how do I add homepage to Django sitemap?
python,django,sitemap,django-sitemaps
Create a sitemap for static views: class StaticViewSitemap(sitemaps.Sitemap): priority = 0.5 changefreq = 'daily' def items(self): return ['home'] def location(self, item): return reverse(item) This assumes you have url pattern for the homepage with name "home" url(r'^$', views.homepage, name="home"), Then add the StaticViewSitemap to the sitemaps dict in your urls.py. sitemaps...
Django sitemap - double http:// in front of URL
You have probably included http:// in your Site object's domain name from the sites framework (django.contrib.sites). Remove it. This field should only include the actual domain name, not the protocol, as the protocol itself can change (e.g. to https://). ...
escaped_fragment in sitemap or shebang?
javascript,ajax,seo,sitemap,shebang
as outlined in google's specification under 'Role of the Search Engine Crawler' it states The search engine agrees to display in the search results the corresponding pretty URLs: thus, http://www.example.com/#!veryAwsomeDynamicPage is displayed in the search results while google fetches the content on http://www.example.com/?_escaped_fragment=veryAwsomeDynamicPage so http://www.example.com/#!veryAwsomeDynamicPage shall be in the sitemap.xml...
404 after attempting to implement Django(1.5.4) Sitemaps
If you are using the Django development server, it should reload when you make any code changes. However in production (e.g. Apache or Nginx) you have to reload the server after making any changes to .py files....
What should be the name of the sitemap file for Google SEO?
basically the site map file name is if XML then /sitemap.xml but its not any issue if you use any other name but we have to add this to webmaster....
Pyramid: How to autogenerate sitemap?
Given you want to establish http://example.com/sitemap.xml as your sitemap URL do that. Add this line to init.py to register URL pattern http://example.com/sitemap.xml as route sitemap config.add_route('sitemap', '/sitemap.xml') register view code for route sitemap and render response with your custom jinja2 template sitemap.jinja2. The file extension `jinja2' will trigger usage of...
How to evidence a particular page on Google SERP?
seo,sitemap,googlebot,google-sitemap
These are called sitelinks and they are unrelated to sitemaps. Google only shows them when: It understands the structure of your website (typically via the structures in URLs). It trusts your website's content (no spam). The content/link is relevant and useful for the corresponding user query. Some say implementing breadcrumbs...
Presta Sitemap Bundle Install Error
php,xml,symfony2,composer-php,sitemap
Try to avoid installing branches! Their stability is always "dev", and you cannot be sure which version you are installing or updating to. "presta/sitemap-bundle" has several tagged releases according to Packagist, currently ranging from 1.0.0 to 1.4.0. Pick one of these versions (there is nothing wrong with using the latest...
XML Sitemap Parsing Error With PHP
In XML ampersands has to be replaced by entity, use & instead. <loc>http://fulldistortion.co.uk/post.php?id=$post_id&title=$post_url_title</loc> ^^^^^ ...
Google sitemap namespace confusion
Problem solved by replace add the xhtml namespace to all the 'link' tags. Replacing <link with <xhtml:link ...
Add SimpleXMLElement Object to Array
php,arrays,object,simplexml,sitemap
$page is not an array (or an array-object that won't mean anything), it is an object (an instance of the class). So you can't use the array methods with it. You can only use available simpleXMLElement class methods. For your particular needs, simpleXMLElement doesn't offer any methods to append as...
urlset tag issue when creating sitemap.xml using classic ASP
xml,asp-classic,sitemap,xmldom
Try using CreateNode rather than CreateElement. If you pass the namespace as the third argument, the xmlns attribute isn't added to the element anymore. Something like this works for me: Dim theDom, theRoot, theParent,theChild, theID, docInstruction const NODE_ELEMENT = 1 const xmlns = "http://www.sitemaps.org/schemas/sitemap/0.9" Set theDom = Server.CreateObject("Microsoft.XMLDOM") Set theRoot...
PHP Create sitemap by crawling a given site [closed]
This one does work, I tested it on my server http://www.plop.at/en/php.html
cannot create google sitemap for Magento 2nd storefront
You need to create Website subfolder in sitemap folder. For example: showcarsign.com/carsign/sitemap.xml showcarboards.com/carboards/sitemap.xml You can read more in detail from the following links: http://www.freshwebservices.com/blog/107-magento-multi-domain-sitemaps https://oldwildissue.wordpress.com/2012/02/27/magento-google-sitemap-generation-for-multistore-installation/ Hope this help!!...
Creating sitemap for more than 50000 urls in Django
python,django,sitemap,django-sitemaps
From the documentation: The sitemap framework also has the ability to create a sitemap index that references individual sitemap files, one per each section defined in your sitemaps dictionary. You can use the same sitemaps object you have and pass it to django.contrib.sitemaps.views.index(), it will refer to the the individual...
google sitemap for tx_news records with dd_googlesitemap_dmf (or alternative)
I had to downgrade dd_googlesitemap (using 1.2.0 now) to get dd_googlesitemap_dmf to work
XML Sitemaps: best practice when listing 'alternatives'?
In other words, is the first entry nonsense and needs to be removed? No, keep it. and do I need to specify every language as an alternative to every other language on the site? Unfortunately, yes. That has been explicitly recommended in Google Webmasters Hangouts....
Use `robots.txt` in a multilingual site
You need to put one robots.txt at the top level. The robots.txt file must be in the top-level directory of the host, accessible though the appropriate protocol and port number. https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt...
Parsing the urls in sitemap with different url format using sitemap spider in scrapy, python
python,web-scraping,scrapy,sitemap,scrapy-spider
I think the nicest and cleanest solution would be to add a downloader middleware which changes the malicious URLs without the spider noticing. import re import urlparse from scrapy.http import XmlResponse from scrapy.utils.gz import gunzip, is_gzipped from scrapy.contrib.spiders import SitemapSpider # downloader middleware class SitemapWithoutSchemeMiddleware(object): def process_response(self, request, response, spider):...
Is it necesary to generate sitemaps for old indexed urls?
If the URLs are active and important from ranking perspective, it is better to have it in sitemap even if it is indexed already. It helps to provide metadata information like when page was last changed, how often it is changed etc.
ASP.NET sitemap , how important?
Site Map Types There are 3 different kinds of "site maps". A page that has links on it that assists users by giving them a complete outline of the site structure. This sometimes, but not always, follows the site hierarchy. Also, it is usually limited to major categories of the...
Which URL should be in sitemap.xml when redirecting request
php,seo,sitemap,url-redirection
Sitemaps (as specified on sitemaps.org) are used "to inform search engines about pages on their sites that are available for crawling". So always add those (canonical) URLs that search engines may crawl and index. They can’t crawl http://www.example.com/, as it’s no page in the first place, so you should add...
Magento: Overwrite Sitemap model class function
Rewrite correctly the class you override in app/code/local/Yourcompany/Yourmodule/etc/config.xml <config> <global> <models> <sitemap_resource> <rewrite> <cms_page>Yourcompany_Yourmodulename_Sitemap_Model_Resource_Cms_Page</cms_page> </rewrite> </sitemap_resource> </models> </global> </config> and then in app/code/local/Yourcompany/Yourmodule/Sitemap/Model/Resource/Cms/Page.php you must extend like this: class...
Sitemap with laravel
It works when I used: {{'<?xml version="1.0" encoding="UTF-8" ?>'}} and I updated my controller as: class SitemapController extends BaseController { public function index() { $content = View::make('sitemap'); return Response::make($content)->header('Content-Type', 'text/xml;charset=utf-8'); } } ...
Regex Notepad ++: How to remove everything except url?
If your desired result is like this: http://mywebsite.com/article1 http://mywebsite.com/article2 http://mywebsite.com/article3 search for: \h*<url\b.*?(http[^<]+).*?</url>|<.*?>\s* and replace with captured url (captured in first parenthesized group) \1 \h matches any horzintal space, [^<]+ matches one or more characters, that are not < Be sure to check the checkbox . matches \r and \n...
How to increse number of sitemapindex
For Google, a sitemap can contain references to other sitemaps, but only with one cascading level. So sitemapindex to destinatieTag.xml is fine, but destinatieTag.xml to myUrlXML.xml is not. There can be up to 50 000 URLs in sitemapindex pointing to other sitemaps. Those sitemaps can each contain 50 000 URLs...
Search engine index issue [closed]
indexing,seo,sitemap,google-search,google-adwords
You have a noindex tag on your pages <meta name="robots" content="noindex"> Remove that....
Microsoft Azure SqlSiteMapProvider Issue
sql,azure,sitemap,provider,sqlsitemapprovider
Solved. The problem is in Azure on custom sitemap types we have to put the full namespace. At the old website We have this: <add name="AspNetSqlSiteMapProvider" type="SqlSiteMapProvider" ... For Azure We need to put it like this: <add name="AspNetSqlSiteMapProvider" type="Site_Base.SqlSiteMapProvider" ... ...
MvcSiteMapProvider is using ascx
asp.net,asp.net-mvc,asp.net-mvc-5,sitemap,mvcsitemapprovider
When the NuGet package detects no files with a .aspx extension in your project or detects any files with a .cshtml or a .vbhtml extension, it will install the .cshtml templates. Unfortunately, there is no reasonable default when both .aspx and .cshtml extensions are detected or no files with the...
TYPO3 Extbase build own Sitemap
If you use ts for create sitemap you can add any items. http://www.adick.at/2010/06/01/typoscript-xml-sitemap/. MB this will help 20 = CONTENT 20 { table = tx_adprojects_domain_model_project select { orderBy = title ASC languageField = sys_language_uid pidInList = 11,12,13,14,15,30 } renderObj = COA renderObj { stdWrap.wrap = <url>|</url> 5 = TEXT 5...
Disable urlencode in sitemap_generator gem
ruby,ruby-on-rails-3,gem,sitemap
This isn't possible - a naked ampersand would be invalid xml. Literal ampersands in xml have to be replaced with &
'Distinct' entries only to create a sitemap.xml file using Django (1.5.4)
The problem is in your items method, not your location method. The items method is what generates the list of items in the site map, the location method is just supposed to return one location given the one item it is passed as its obj parameter. So try this: def...
Google snippet tree structure
sitemap,rich-snippets,google-rich-snippets
So, the keyword I was looking for is "Google Sitelinks". Here as some documentations about it : https://support.google.com/webmasters/answer/47334?hl=en https://developers.google.com/structured-data/slsb-overview http://bloggerspassion.com/google-site-links-how-to-get-google-sitelinks-for-your-website/ Thanks to @Mark....
What should be the name of sitemap file while submitting to Google Webmasters?
sitemap,google-webmaster-tools
It is not obligatory. you can use any name for your sitemap and you can place it anywhere. but it is just for Google. if you want to all of search engines find your sitemap, you should name it "sitemap.xml" and place it in your root directory....
How do I set main URL a priority of 1.0 and subdirectory urls priority of 0.5 in sitemap.xml
php,xml,sitemap,priority,sitemap.xml
I'm answering my own question because I figured it out. I simply did this. I took this line of code and expanded it with an if and else statement. went from this $url -> appendChild( $priority = $xml->createElement('priority', '0.5') ); to this if($file['name'] != '/') { $url -> appendChild( $priority...
Django sitemap index pagination URL format
You're missing a \ in your url. url(r'^sitemap-(?P<section>.+).xml$','django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps}), should be url(r'^sitemap-(?P<section>.+)\.xml$','django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps}), ...
Redirects required for pages no longer in sitemap
.htaccess,redirect,seo,sitemap,google-webmaster-tools
It is better to have 301 redirect for some time(month or two) even though you can change all your links to nonphp urls. This way any residual URLs(will always be there) that are hanging out there will be taken care and google will index nonphp urls from your 301 redirect....