某业务系统不能访问，weblogic控制台登录不了，查看系统资源发现CPU使用率100%
[root@app ~]# top
top - 11:38:40 up 17 days, 20:41, 1 user, load average: 7.99, 7.82, 7.63
Tasks: 171 total, 1 running, 170 sleeping, 0 stopped, 0 zombie
Cpu(s):100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16436780k total, 13225904k used, 3210876k free, 275464k buffers
Swap: 16779884k total, 0k used, 16779884k free, 2676416k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5419 root 25 0 10.5g 9.6g 43m S 791.8 61.0 2549:30 java
7931 root 15 0 12740 1152 820 R 0.3 0.0 0:00.01 top
1 root 15 0 10348 692 584 S 0.0 0.0 0:01.99 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.08 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.13 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.15 migration/1
5 root 34 19 0 0 0 S 0.0 0.0 0:00.30 ksoftirqd/1
6 root RT -5 0 0 0 S 0.0 0.0 0:00.17 migration/2
7 root 34 19 0 0 0 S 0.0 0.0 0:00.30 ksoftirqd/2
8 root RT -5 0 0 0 S 0.0 0.0 0:00.15 migration/3
9 root 34 19 0 0 0 S 0.0 0.0 0:00.28 ksoftirqd/3
10 root RT -5 0 0 0 S 0.0 0.0 0:00.15 migration/4
11 root 34 19 0 0 0 S 0.0 0.0 0:00.28 ksoftirqd/4
12 root RT -5 0 0 0 S 0.0 0.0 0:00.14 migration/5
13 root 34 19 0 0 0 S 0.0 0.0 0:00.27 ksoftirqd/5
14 root RT -5 0 0 0 S 0.0 0.0 0:00.15 migration/6
15 root 34 19 0 0 0 S 0.0 0.0 0:00.31 ksoftirqd/6
16 root RT -5 0 0 0 S 0.0 0.0 0:00.70 migration/7
17 root 34 19 0 0 0 S 0.0 0.0 0:00.21 ksoftirqd/7
18 root 10 -5 0 0 0 S 0.0 0.0 7:31.65 events/0
19 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/1
20 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/2
21 root 10 -5 0 0 0 S 0.0 0.0 0:00.02 events/3
22 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/4
23 root 10 -5 0 0 0 S 0.0 0.0 0:00.02 events/5
24 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/6
25 root 10 -5 0 0 0 S 0.0 0.0 0:00.03 events/7
26 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
163 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
174 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
175 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/1
176 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/2
177 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/3
178 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/4
179 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/5
180 root 10 -5 0 0 0 S 0.0 0.0 0:00.04 kblockd/6
181 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/7
查看weblogic日志找到发下信息:
[root@app1 base_domain]# more weblogic.log <Dec 15, 2016 9:09:19 AM CST> <Critical> <Server> <BEA-002616> <Failed to listen on channel "Default" on 10.138.130.61:7001, failure count: 1, failing for 0 seconds, java.io.IOException: Too many open files>
问题原因是因为对服务器配置的允许打开文件的最大数量比weblogic所需要的数量小而造成的。
解决方法:
1.设置 ulimit -u 65535
2.修改/etc/security/limits.conf在文件末加上
soft nofile 65535
hard nofile 65535
参考:System Crashes With “java.net.SocketException: Too many open files” Error (Doc ID 1194575.1)