.htaccess - 如何修复 ".htaccess RewriteRule results in duplicated pages"

Question

再会。
我们有一个用普通 PHP 编码的门户网站，其中有一个博客部分[mysite.com/blog.php?blog=1]输出所需文件的内容。
这导致我们的 SEO 专家指出这是一个格式错误的 SEO 网址。
然后我们决定使用 .htaccess 来显示命名的 URLblog=Residential_Relocation -> blogs.php?blog=1输出 [mysite.com/blog.php?blog=Residential_Relocation]但现在它被视为重复。
我们如何才能只从 blog=1 URL 读取文件而不被爬虫抓取？

php_value session.cookie_lifetime 18000
php_value session.gc_maxlifetime 18000

RewriteEngine On 
RewriteCond %{HTTPS} off 
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

<IfModule mod_deflate.c>
  # Compress HTML, CSS, JavaScript, Text, XML and fonts
  AddOutputFilterByType DEFLATE application/javascript
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
  AddOutputFilterByType DEFLATE application/x-font
  AddOutputFilterByType DEFLATE application/x-font-opentype
  AddOutputFilterByType DEFLATE application/x-font-otf
  AddOutputFilterByType DEFLATE application/x-font-truetype
  AddOutputFilterByType DEFLATE application/x-font-ttf
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE font/opentype
  AddOutputFilterByType DEFLATE font/otf
  AddOutputFilterByType DEFLATE font/ttf
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE image/x-icon
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE text/html
  AddOutputFilterByType DEFLATE text/javascript
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml

  # Remove browser bugs (only needed for really old browsers)
  BrowserMatch ^Mozilla/4 gzip-only-text/html
  BrowserMatch ^Mozilla/4\.0[678] no-gzip
  BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
  Header append Vary User-Agent
</IfModule>

<IfModule mod_expires.c>
  ExpiresActive On

  # Images
  ExpiresByType image/jpeg "access plus 1 year"
  ExpiresByType image/gif "access plus 1 year"
  ExpiresByType image/png "access plus 1 year"
  ExpiresByType image/webp "access plus 1 year"
  ExpiresByType image/svg+xml "access plus 1 year"
  ExpiresByType image/x-icon "access plus 1 year"

  # Video
  ExpiresByType video/webm "access plus 1 year"
  ExpiresByType video/mp4 "access plus 1 year"
  ExpiresByType video/mpeg "access plus 1 year"

  # Fonts
  ExpiresByType font/ttf "access plus 1 year"
  ExpiresByType font/otf "access plus 1 year"
  ExpiresByType font/woff "access plus 1 year"
  ExpiresByType font/woff2 "access plus 1 year"
  ExpiresByType application/font-woff "access plus 1 year"

  # CSS, JavaScript
  ExpiresByType text/css "access plus 1 month"
  ExpiresByType text/javascript "access plus 1 month"
  ExpiresByType application/javascript "access plus 1 month"

  # Others
  ExpiresByType application/pdf "access plus 1 month"
  ExpiresByType image/vnd.microsoft.icon "access plus 1 year"
</IfModule>

# BEGIN cPanel-generated php ini directives, do not edit
# Manual editing of this file may result in unexpected behavior.
# To make changes to this file, use the cPanel MultiPHP INI Editor (Home >> Software >> MultiPHP INI Editor)
# For more information, read our documentation (https://go.cpanel.net/EA4ModifyINI)
<IfModule php7_module>
   php_flag display_errors On
   php_value max_execution_time 300
   php_value max_input_time 600
   php_value max_input_vars 10000
   php_value memory_limit 320M
   php_value post_max_size 80M
   php_value session.gc_maxlifetime 14400
   php_value session.save_path "/var/cpanel/php/sessions/ea-php74"
   php_value upload_max_filesize 20M
   php_flag zlib.output_compression Off
</IfModule>
<IfModule lsapi_module>
   php_flag display_errors On
   php_value max_execution_time 300
   php_value max_input_time 600
   php_value max_input_vars 10000
   php_value memory_limit 320M
   php_value post_max_size 80M
   php_value session.gc_maxlifetime 14400
   php_value session.save_path "/var/cpanel/php/sessions/ea-php74"
   php_value upload_max_filesize 20M
   php_flag zlib.output_compression Off
</IfModule>
# END cPanel-generated php ini directives, do not edit

# php -- BEGIN cPanel-generated handler, do not edit
# Set the “ea-php74” package as the default “PHP” programming language.
<IfModule mime_module>
  AddHandler application/x-httpd-ea-php74___lsphp .php .php7 .phtml
</IfModule>
# php -- END cPanel-generated handler, do not edit

<Files 403.shtml>
order allow,deny
allow from all
</Files>

deny from 45.129.137.249
deny from 43.242.128.31
deny from 104.131.124.162
deny from 216.144.247.254

RewriteRule ^blog=Professional_Packing_and_Wrapping_Services/?$ blogs.php?blog=1 [NC]
RewriteRule ^blog=GIT,_All_Risk_and_General_Insurance/?$ blogs.php?blog=2 [NC]
RewriteRule ^blog=Relocation_Station/?$ blogs.php?blog=4 [NC]

RewriteRule ^category=Residential_Relocation/?$ blogs.php?cat=1 [NC]
RewriteRule ^category=Commercial_Relocation/?$ blogs.php?cat=2 [NC]
RewriteRule ^category=International_Relocation/?$ blogs.php?cat=3 [NC]
RewriteRule ^category=Pet_Transport/?$ blogs.php?cat=4 [NC]
RewriteRule ^category=Car_Transport/?$ blogs.php?cat=5 [NC]
RewriteRule ^category=Wrapping_Services/?$ blogs.php?cat=6 [NC]
RewriteRule ^category=Packing_Services/?$ blogs.php?cat=7 [NC]
RewriteRule ^category=Courier_Services/?$ blogs.php?cat=8 [NC]
RewriteRule ^category=Cleaning_Services/?$ blogs.php?cat=9 [NC]
RewriteRule ^category=Storage/?$ blogs.php?cat=10 [NC]
RewriteRule ^category=Insurance/?$ blogs.php?cat=11 [NC]
RewriteRule ^category=Moving_Tips/?$ blogs.php?cat=12 [NC]
RewriteRule ^category=General/?$ blogs.php?cat=13 [NC]

RewriteRule ^keyword=relocation/?$ blogs.php?key=1 [NC]
RewriteRule ^keyword=residential/?$ blogs.php?key=2 [NC]
RewriteRule ^keyword=commercial/?$ blogs.php?key=3 [NC]
RewriteRule ^keyword=international/?$ blogs.php?key=4 [NC]
RewriteRule ^keyword=storage/?$ blogs.php?key=5 [NC]
RewriteRule ^keyword=car/?$ blogs.php?key=6 [NC]
RewriteRule ^keyword=vehicle/?$ blogs.php?key=7 [NC]
RewriteRule ^keyword=pet/?$ blogs.php?key=8 [NC]
RewriteRule ^keyword=insurance/?$ blogs.php?key=9 [NC]
RewriteRule ^keyword=packers/?$ blogs.php?key=10 [NC]
RewriteRule ^keyword=packing/?$ blogs.php?key=11 [NC]
RewriteRule ^keyword=courier/?$ blogs.php?key=12 [NC]
RewriteRule ^keyword=wrapping/?$ blogs.php?key=13 [NC]
RewriteRule ^keyword=cleaning/?$ blogs.php?key=14 [NC]
RewriteRule ^keyword=long_distance/?$ blogs.php?key=15 [NC]
RewriteRule ^keyword=transport/?$ blogs.php?key=16 [NC]
RewriteRule ^keyword=moving/?$ blogs.php?key=17 [NC]
RewriteRule ^keyword=affordable/?$ blogs.php?key=18 [NC]
RewriteRule ^keyword=professional/?$ blogs.php?key=19 [NC]
RewriteRule ^keyword=cost_effective/?$ blogs.php?key=20 [NC]
RewriteRule ^keyword=tips/?$ blogs.php?key=21 [NC]

Answer 1

要消除 SEO 的重复惩罚，您可以阻止所有指向内部 URI 的 URL，即 /blogs.php?blog=<number> , /blogs.php?cat=<number>和 /blogs.php?key=<number> .
在 RewriteEngine On 正下方插入一条新规则线:

RewriteEngine On

# prohibit direct access to internal URIs
RewriteCond %{THE_REQUEST} /blogs\.php\?(blog|cat|key)=\d+ [NC]
RewriteRule ^ - [F]

# existing rules go below this line