GitLab 502问题的解决
问题:
502 Whoops, GitLab is taking too much time to respond.
日志:
[root@cs12-66-gitlab ~]# my gitlab-ctl tail
-bash: my: command not found
[root@cs12-66-gitlab ~]# gitlab-ctl tail
==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==
# Logfile created on 2018-02-02 14:34:26 +0800 by logger.rb/56438==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2018-02-02-14-34-39.log <==-> 0.3182s== Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/001_admin.rb
Administrator account created:login: root
password: You'll be prompted to create one on your first visit.== Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/010_settings.rb==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/" for 10.100.16.218 at 2018-02-02 14:54:06 +0800
Started GET "/-/metrics" for 127.0.0.1 at 2018-02-02 14:54:06 +0800
Processing by MetricsController#index as HTML
Completed 200 OK in 3ms (Views: 0.8ms | ActiveRecord: 0.0ms)
Processing by RootController#index as HTML
Redirected to http://git.tuandai888.com/users/sign_in
Filter chain halted as :redirect_unlogged_user rendered or redirected
Completed 302 Found in 9ms (ActiveRecord: 0.0ms)
Started GET "/users/sign_in" for 10.100.16.218 at 2018-02-02 14:54:10 +0800
Processing by SessionsController#new as HTML==> /var/log/gitlab/gitlab-rails/grpc.log <==
# Logfile created on 2018-02-02 14:41:50 +0800 by logger.rb/56438==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":3.63,"view":0.5,"db":0.0,"time":"2018-02-02T06:52:06.991Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":4.42,"view":0.96,"db":0.0,"time":"2018-02-02T06:52:21.998Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":17.59,"view":13.91,"db":0.0,"time":"2018-02-02T06:52:38.632Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":4.71,"view":1.08,"db":0.0,"time":"2018-02-02T06:52:52.001Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":3.6,"view":0.55,"db":0.0,"time":"2018-02-02T06:53:06.992Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":4.08,"view":0.54,"db":0.0,"time":"2018-02-02T06:53:21.984Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":3.64,"view":0.42,"db":0.0,"time":"2018-02-02T06:53:36.985Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":4.94,"view":0.73,"db":0.0,"time":"2018-02-02T06:53:51.985Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"duration":4.72,"view":0.84,"db":0.0,"time":"2018-02-02T06:54:06.991Z","params":{},"remote_ip":null,"user_id":null,"username":null}
{"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":302,"duration":10.83,"view":0.0,"db":0.0,"location":"http://git.tuandai888.com/users/sign_in","time":"2018-02-02T06:54:10.479Z","params":{},"remote_ip":"10.100.16.218","user_id":null,"username":null}==> /var/log/gitlab/gitlab-rails/api_json.log <==
# Logfile created on 2018-02-02 14:41:51 +0800 by logger.rb/56438==> /var/log/gitlab/gitlab-rails/application.log <==
# Logfile created on 2018-02-02 14:42:32 +0800 by logger.rb/56438
February 02, 2018 14:42: User "Administrator" (admin@example.com) was created==> /var/log/gitlab/logrotate/current <====> /var/log/gitlab/unicorn/current <==
2018-02-02_06:42:37.56441 starting new unicorn master
2018-02-02_06:49:48.90547 adopted new unicorn master 3906==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
I, [2018-02-02T14:49:47.581121 #4594] INFO -- : worker=0 ready
I, [2018-02-02T14:49:47.612613 #4597] INFO -- : worker=1 ready
I, [2018-02-02T14:49:47.674536 #4600] INFO -- : worker=2 ready
I, [2018-02-02T14:49:47.710865 #4603] INFO -- : worker=3 ready
I, [2018-02-02T14:49:47.756704 #4606] INFO -- : worker=4 ready
I, [2018-02-02T14:49:47.825869 #3906] INFO -- : master process ready
I, [2018-02-02T14:49:47.836401 #4609] INFO -- : worker=5 ready
I, [2018-02-02T14:49:47.887782 #4612] INFO -- : worker=6 ready
I, [2018-02-02T14:49:47.909891 #4615] INFO -- : worker=7 ready
I, [2018-02-02T14:49:47.940642 #4618] INFO -- : worker=8 ready==> /var/log/gitlab/unicorn/unicorn_stdout.log <====> /var/log/gitlab/sidekiq/current <==
2018-02-02_06:52:59.51350 127.0.0.1 - - [02/Feb/2018:14:52:59 CST] "GET /metrics HTTP/1.1" 200 25
2018-02-02_06:52:59.51358 - -> /metrics
2018-02-02_06:53:14.51410 127.0.0.1 - - [02/Feb/2018:14:53:14 CST] "GET /metrics HTTP/1.1" 200 25
2018-02-02_06:53:14.51417 - -> /metrics
2018-02-02_06:53:29.51384 127.0.0.1 - - [02/Feb/2018:14:53:29 CST] "GET /metrics HTTP/1.1" 200 25
2018-02-02_06:53:29.51390 - -> /metrics
2018-02-02_06:53:44.51387 127.0.0.1 - - [02/Feb/2018:14:53:44 CST] "GET /metrics HTTP/1.1" 200 25
2018-02-02_06:53:44.51392 - -> /metrics
2018-02-02_06:53:59.51637 127.0.0.1 - - [02/Feb/2018:14:53:59 CST] "GET /metrics HTTP/1.1" 200 25
2018-02-02_06:53:59.51640 - -> /metrics==> /var/log/gitlab/gitlab-workhorse/current <==
2018-02-02_06:46:40.87969 10.100.12.66 @ - - [2018-02-02 14:46:40.877704109 +0800 CST] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 0.001910
2018-02-02_06:46:40.89513 2018/02/02 14:46:40 Send static file "/opt/gitlab/embedded/service/gitlab-rails/public/favicon.ico" ("") for GET "/favicon.ico"
2018-02-02_06:46:40.89566 10.100.12.66 @ - - [2018-02-02 14:46:40.89417661 +0800 CST] "GET /favicon.ico HTTP/1.1" 200 5430 "http://10.100.12.66/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 0.001187
2018-02-02_06:47:25.68625 2018/02/02 14:47:25 error: GET "/": badgateway: failed after 0s: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory
2018-02-02_06:47:25.68697 git.tuandai888.com @ - - [2018-02-02 14:47:25.683237091 +0800 CST] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 0.003579
2018-02-02_06:47:25.87230 2018/02/02 14:47:25 Send static file "/opt/gitlab/embedded/service/gitlab-rails/public/favicon.ico" ("") for GET "/favicon.ico"
2018-02-02_06:47:25.87285 git.tuandai888.com @ - - [2018-02-02 14:47:25.871579483 +0800 CST] "GET /favicon.ico HTTP/1.1" 200 5430 "http://git.tuandai888.com/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 0.001192
2018-02-02_06:47:45.63429 2018/02/02 14:47:45 error: GET "/": badgateway: failed after 0s: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory
2018-02-02_06:47:45.63501 git.tuandai888.com @ - - [2018-02-02 14:47:45.631742962 +0800 CST] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 0.003081
2018-02-02_06:54:10.49117 git.tuandai888.com @ - - [2018-02-02 14:54:06.631802992 +0800 CST] "GET / HTTP/1.1" 302 105 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 3.859047==> /var/log/gitlab/nginx/current <====> /var/log/gitlab/nginx/error.log <====> /var/log/gitlab/nginx/gitlab_access.log <==
10.100.16.218 - - [02/Feb/2018:14:46:40 +0800] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:46:40 +0800] "GET /favicon.ico HTTP/1.1" 200 5430 "http://10.100.12.66/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:46:40 +0800] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:46:40 +0800] "GET /favicon.ico HTTP/1.1" 200 5430 "http://10.100.12.66/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:46:40 +0800] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:46:40 +0800] "GET /favicon.ico HTTP/1.1" 200 5430 "http://10.100.12.66/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:47:25 +0800] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:47:25 +0800] "GET /favicon.ico HTTP/1.1" 200 5430 "http://git.tuandai888.com/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:47:45 +0800] "GET / HTTP/1.1" 502 2916 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
10.100.16.218 - - [02/Feb/2018:14:54:10 +0800] "GET / HTTP/1.1" 302 105 "" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
安装好GitLab,开启服务,发现有502错误。这是开始寻找解决办法,各种百度。
1.找到/var/log/gitlab/nginx
中的错误日志文件,发现有如下错误/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket failed (2: No such file or directory)
,然后用 nc命令创建了这个socket文件,最终权限设为srwxrwxrwx
,用户和组设置为git:git,但发现这个方法行不通。
2.这时我跑到GitLab的官网去寻找解决办法,https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
ctrl+f 502 找到官方教程中说502出现的问题
Note that on a single-core server it may take up to a minute to restart Unicorn and Sidekiq. Your GitLab instance will give a 502 error until Unicorn is up again.
It is also possible to start, stop or restart individual components.
sudo gitlab-ctl restart sidekiq
Unicorn supports zero-downtime reloads. These can be triggered as follows:
sudo gitlab-ctl hup unicorn
Note that you cannot use a Unicorn reload to update the Ruby runtime.
尝试用上面两个命令解决,发现没有用。
这时我怒了,不断的输入gitlab-ctl status
,发现unicorn的pid一直在变大。而其他几个服务的pid没有变化。
3.这时差不多找到了问题的所在了,应该就是unicorn的问题。然后看官方教程,可以使用gitlab-ctl tail unicorn
来跟踪unicorn的状态,这时候悲催的发现原来时权限不够,chmod -R 777 /var/log/gitlab
502 Whoops, GitLab is taking too much time to respond。经搜索梳理网上几篇相同问题的解决方案,归纳原因可能如下:
1、unicorn原8080默认端口被容器中别的进程已经占用,必须调整为没用过的
2、gitlab的timeout设置过小,默认为60
从这两个原因着手,开始调整配置:
1、使用以下命令编辑gitlab配置文件:
sudo docker exec -it gitlab vim /etc/gitlab/gitlab.rb
经试验,应该修改以下配置项:
unicorn['port'] = 8888
gitlab_workhorse['auth_backend'] = "http://localhost:8888"
注意:unicorn['port']与gitlab_workhorse['auth_backend']的端口必须相同
2、调整timout时长,从60秒改为90秒
gitlab_rails['webhook_timeout'] = 90
gitlab_rails['git_timeout']=90
以上配置修改完成后,使用以下命令重启容器:
docker restart gitlab
docker logs gitlab
也可以通过以下命令进入容器,查看gitlab启动状态和动态日志:
sudo docker exec -it gitlab /bin/bash
cd /usr/bin
gitlab-ctl status
gitlab-ctl tail
3、初始镜像、容器的创建步骤是这样的:
sudo docker pull gitlab/gitlab-ce
chmod 777 /var/log/gitlab
sudo docker run --detach --hostname devops --publish 443:443 --publish 80:80 --publish 20022:22 --name gitlab --env GITLAB_OMNIBUS_CONFIG="external_url 'http://devops/'; gitlab_rails['lfs_enabled'] = true;" --restart always --volume /home/linyu/gitlab/config:/etc/gitlab --volume /home/linyu/gitlab/logs:/var/log/gitlab --volume /home/linyu/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest
4、也可以进入容器后,根据容器创建时映射的--volume目录,在里面修改配置,然后重启gitlab:
gitlab-ctl reconfigure
gitlab-ctl retart
gitlab-ctl status
gitlab-ctl tail
转载于:https://www.cnblogs.com/linkenpark/p/8405327.html
GitLab 502问题的解决相关推荐
- 常见的Nginx 502 Bad Gateway解决办法
常见的Nginx 502 Bad Gateway解决办法如下: Nginx 502错误情况1: 网站的访问量大,而php-cgi的进程数偏少. 针对这种情况的502错误,只需增加php-cgi的进程数 ...
- php502bad gateway,PHP表单提交后出现502 bad gateway解决办法
搜索热词 <PHP表单提交后出现502 bad gateway解决办法>要点: 本文介绍了PHP表单提交后出现502 bad gateway解决办法,希望对您有用.如果有疑问,可以联系我们 ...
- 502网关错误解决_什么是502错误网关错误(以及如何解决)?
502网关错误解决 A 502 Bad Gateway Error occurs when you try to visit a web page, but one web server gets a ...
- Gitlab出现502报错解决
自己在服务器上搭建的gitlab,出现502错误, 解决方案: 尝试 gitlab-ctl reconfigure .gitlab-ctl restart 如果这两个命令无效 则参考下列解决方案 常见 ...
- CentOS 7搭建GitLab服务器踩坑——解决Whoops, GitLab is taking too much time to respond 502 错误
问题描述 按照官方教程GitLab Installation一步步操作下来,成功下载和安装了GitLab,相关配置也改了,external_url改成了服务器的ip和开放的端口号,端口号在防火墙上也开 ...
- 解决GitLab 502
1.查看Nginx日志 fztree@x2:/var/log$ less /var/log/nginx/gitlab_error.log 2015/09/06 03:27:31 [error] 282 ...
- IIS 7.x Application Request Routing(ARR) 502错误的解决方法
IIS ARR可实现基于反向代理的Server Farms,默认配置会导致在需要长时间操作超时的502.3错误,具体解释可以参考http://blogs.iis.net/richma/archive/ ...
- lnmp一键包502 Bad Gateway解决方法
使用LNmp有一段时间了,可是在流量大的时候,有时会莫名其妙的出现Nginx 502 Bad Gateway,不是php-fpm死掉了,就是nginx死掉了,网站多的时候根本没空每天一个个检查,往往是 ...
- LNMPA(LNMP0.7)安装出现502 Bad Gateway解决方法
2019独角兽企业重金招聘Python工程师标准>>> LNMPA是什么? 简单的说就是Linux+Nginx+MySql+PHP+Apache 怎么会用这种构架? Nginx和Ap ...
最新文章
- android.view.InflateException: Binary XML file line #7: Binary XML file line #7
- XCODE 6.1.1 配置GLFW
- bzoj1207(HNOI2004)打鼹鼠
- Android应用开发:数据存储和界面展现-1
- 机器学习:决策树及ID3,C4.5,CART算法描述
- CentOS7—HAProxy安装与配置
- Java和ABAP中的几种引用类型的分析和比较
- C++(STL):19---deque之删除和emplace用法
- 上世纪八九十年代的收录放音机拆解
- regex flag
- 年终感想——财务自由的程序员,你见过吗?
- java excel 设置列为日期,POI设置Excel单元格格式 (数值,日期,文本等等)
- java js css 压缩工具_JSCSS压缩工具YUICompressor
- 晒晒那些我在云栖社区获得的奖品
- 基于python的opencv的学习
- 【机器学习】机器学习模型迭代方法(Python)
- 开关磁阻电机控制仿真 开关磁阻电机传统控制:电流斩波控制、电压PWM控制、角度位置控制。 智能控制:12/8三相开关磁阻电机有限元分析本体建模
- 关于Keil编译程序出现“File has been changed outside the editor,reload?”的解决方法
- python cookie动态请求_Python请求自动处理cookie
- 怎么批量查询快递物流信息