Project

General

Profile

Scénario #10144

to many open files avec l'utilisation de mkstemp

Added by Gwenael Remond over 6 years ago. Updated about 6 years ago.

Status:
Terminé (Sprint)
Priority:
Normal
Assigned To:
-
Category:
-
Start date:
12/24/2014
Due date:
01/23/2015
% Done:

100%

Estimated time:
(Total: 6.00 h)
Spent time:
(Total: 2.33 h)
Story points:
1.0
Remaining (hours):
0.00 hour
Velocity based estimate:

Description

Lors d'un import annuel des élèves, trop de stdout d'ouvert on provoqué une erreur de type Too many open files sur un amon ecole en 2.3

2014-11-21 22:50:55 - ERROR 
Traceback (most recent call last):
  File "/usr/share/ead2/backend/bin/importation.py", line 405, in do_import_eleve
    writer.write_eleve(storage=self.store, connexion=connexion, etab=etab)
  File "/usr/lib/pymodules/python2.6/scribe/importation/writer.py", line 505, in write_eleve
    _maj_eleve(eleve, user, login, etab)
  File "/usr/lib/pymodules/python2.6/scribe/importation/writer.py", line 458, in _maj_eleve
    user._change_classe(login, classe, etab)
  File "/usr/lib/pymodules/python2.6/scribe/eleves.py", line 175, in _change_classe
    self._inscription(user, new_classe, sync=False, etab=new_etab)
  File "/usr/lib/pymodules/python2.6/scribe/eleves.py", line 187, in _inscription
    User._inscription(self, login, groupe, sync=sync, etab=etab)
  File "/usr/lib/pymodules/python2.6/scribe/eoleuser.py", line 374, in _inscription
    tool.launch_smbldap_tool(cmd, num_etab, etabgroup)
  File "/usr/lib/pymodules/python2.6/scribe/eoletools.py", line 470, in launch_smbldap_tool
    res = system_out(cmd, container='fichier', env=env)
  File "/usr/lib/pymodules/python2.6/pyeole/process.py", line 85, in system_out
    cmd = _gen_container_cmd(cmd, container, context, env, pty)
  File "/usr/lib/pymodules/python2.6/pyeole/process.py", line 26, in _gen_container_cmd
    if not test_conteneur(container, ip_address):
  File "/usr/lib/pymodules/python2.6/pyeole/process.py", line 155, in test_conteneur
    if not system_out(cmd)[1].strip().lower().endswith(' is running'):
  File "/usr/lib/pymodules/python2.6/pyeole/process.py", line 100, in system_out
    env=env)
  File "/usr/lib/python2.6/subprocess.py", line 633, in __init__
    errread, errwrite)
  File "/usr/lib/python2.6/subprocess.py", line 1039, in _execute_child
    errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files
2014-11-21 22:50:55 - DEBUG suppression du lock eoleimport

L'ajout du paramètre close_fds=True à subprocess.Popen dans system_out de process.py a pu résoudre le problème, l'import passe sans problème dès que ce paramètre est fixé.

cf la doc officielle python : https://docs.python.org/2/library/subprocess.html#popen-constructor


Subtasks

Tâche #10185: Rechercher et corriger les utilisations de mkstempFerméGwenael Remond

History

#1 Updated by Gwenael Remond over 6 years ago

la solution proposée n'est pas la bonne. L'erreur provient en fait d'un mauvais usage de tempfile.mkstemp()

cf le billet de logilab sur la nécessité d'utiliser os.remove(temp_path)

http://www.logilab.org/blogentry/17873
dans /usr/share/pyshared/script/eoletools.py

from os import system, makedirs, unlink, close #(ajout de close)

fd, tempfile = mkstemp(suffix='.temp', dir='%s%s' % (conteneurs.get('container_path_fichier', ''), confdir))
# ligne 456
close(fd)
# ligne 471, dans  
if tempfile is not None:

#2 Updated by Luc Bourdot over 6 years ago

  • Tracker changed from Anomalie to Scénario
  • Due date set to 01/22/2015
  • Target version set to sprint 2015 2-4

#3 Updated by Fabrice Barconnière over 6 years ago

  • Subject changed from to many open files avec system_out to to many open files avec l'utilisation de mkstemp

#4 Updated by Klaas TJEBBES over 6 years ago

  • Story points set to 1.0

#5 Updated by Joël Cuissinat about 6 years ago

  • Status changed from Nouveau to Terminé (Sprint)

Also available in: Atom PDF