gmane.comp.python.wsgi.uwsgi.general

Re: sharing objects amongst workers

Tony Lambropoulos — 2013-05-21T16:55:15

Hi, a better serialization system is definitely something I am beginnign to
test.  Another solution I am looking into is transforming the data I am
storing into a more compact form, like replacing the strings I have with
integers that represent them (as this portion of the app doesn't need to be
human readable, it just needs to be used to make various decisions), and
this will allow me to store the objects in the cache and get them out
quickly without worring as much about serialization efficiency.


On Mon, May 20, 2013 at 7:43 PM, <uwsgi-request-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org> wrote:

_______________________________________________
uWSGI mailing list
uWSGI-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Re: HTTP router and http-keepalive

André Cruz — 2013-05-21T15:59:34

Meanwhile I think I have discovered a bug. When doing the request manually, pasting the request to a telnet session, the keepalive works. But if I issue the request using cURL, the connection is closed by the server after the response is sent.

In the code this section in http.c is entered:
if (hr->remains > 0) {
hr->session.can_keepalive = 0;

hr->remains is equal to the length of the request. It seems uWSGI thinks I have not read the request but my application has read it and produced a response. When I paste the request manually, hr->remains is <= 0.

An example request:

POST /api/sync/commit_batch HTTP/1.1
Host: xxx
Accept: */*
Authorization: xxx
Content-Length: 316
Content-Type: application/x-www-form-urlencoded

commit_info=eJw9jlFvgjAYRf_K8j3TSKV2yNuGDyAyE8PmkrmQWqpUaGEtxmbG_y4my95ucu_NOV9X2LcdbyxEAB5IW1bSQOR7oAapBEQIe2Dl71_qmRF6KP8v5XuGu_iniXsds5BuU_daT806yfRe6UXxll5SihOaG3fKXz7Wn1V6cts6Wcp2EWTh2WWbhi_z7tCoc1HI-HLcFKvVcRTRTAnbMy5kNWJowElFCUHPM0ERmWGORhpBIcZ0HoRszjmBh91Qj-uJVaxtD7IVdoJ93x8bNgz

Re: HTTP router and http-keepalive

Roberto De Ioris — 2013-05-21T15:14:59


Pipelining is very particular, expecially because it requires allocating
the memory for a new request while the first one is still running. It
opens for dos. This is the main problem (obviously we can limit the
pipeline size, but currently the http/https/spdy router is faster only
because it is simple).

But from a technical point of view, pipelining is supported if the second
request come after the first request has been passed to the backend, and
in my tests, at least chrome and firefox send the request in two different
chunk and 99% of the times, pipelining will automatically work :P

What Does not work is if the client send a single chunk:

"""
GET /one HTTP/1.1
Host: foobar.it

GET /two HTTP/1.1
Host: foobar.it
"""

The second part will be swalloed


The idea of the whole uWSGI project is that your app has the full logic
(this is for allowing ISPs to easily account customers) so thanks to the
internal routing you will be able to selectively add keep-alive.

For example:

error-route-status = 200 addhe

Re: Python Decorators

Alexandro G.S. Mancusi — 2013-05-21T14:52:05

Thank you for the advice. I was proposing the following:

def clientToRedis(redis):
    while True:
        msg = uwsgi.websocket_recv()
        redis.publish('foobar', msg)

def redisToClient(channel):
    while True:
        for msg in channel.listen():
            uwsgi.websocket_send(msg.get('data'))

I wanted to spawn both functions in two Greenlets but uwsgi won't let me. UWSGI is only callable form the main callable.
The first listens to the browser and when something comes in it pushes it onto the redis channel foobar.
The second check the channel for messages and pushes them through the websocket.

Since uwsgi won't allow me to call the uwsgi module form within Greenlets how do work around this.

-----Original Message-----
From: uwsgi-bounces-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org [mailto:uwsgi-bounces-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org] On Behalf Of Roberto De Ioris
Sent: 17 May 2013 12:40
To: uWSGI developers and users list
Subject: Re: [uWSGI] Python Decorators



more or less, tha main gre

Re: HTTP router and http-keepalive

André Cruz — 2013-05-21T14:27:00


I've been testing this, and it seems to work most of the time (when it didn't, I think it was a client problem). I just have some questions:

- Pipeline mode is not supported at all by the http plugin, right? I've noticed that uWSGI explicitly disables keepalive and closes the connection after the response when it detects the client has sent more data than the size of the body.
- Right now I'm manually adding the “Connection: Keep-Alive” header through the uWSGI config. But sometimes this will not be true, as in the pipeline case. Can't uWSGI handle this header for me? Only uWSGI knows for sure if it will maintain the connection open or not. There are cases in which I respond early to the client with a 4XX error without reading the whole request from wsgi.input, uWSGI detects this and closes the connection but the header "Keep-Alive" is sent as well which is also not true. Btw, Django and other frameworks won't let me add the "Connection" header since they claim that it the web server responsibility.

Re: Django - SMTP

Łukasz Mierzwa — 2013-05-21T11:01:42

2013/5/21 Venkatraman S <venkat83-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>



Depends on your definition of "easier", celery + celery-django-mail is
ready and easy to setup, spooler needs very little code to get it working.

EDIT - there is also https://github.com/jaysonsantos/django-uwsgi-mail which
should be even easier ;)

Re: Django - SMTP

Daniel Kopka — 2013-05-21T08:19:22

Thanks for help.
I solved the problem my server IP was banned ;)


--
Pozdrawiam,
Daniel Kopka
Tel: 502276071


2013/5/21 Venkatraman S <venkat83-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>

_______________________________________________
uWSGI mailing list
uWSGI-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Re: Access to the config options at the global scopeinuWSGI app

Aarni Koskela — 2013-05-21T06:58:07

Use the documentation, Luke! ;)

http://uwsgi-docs.readthedocs.org/en/latest/PythonModule.html#uwsgi.opt

And to prove it works:

# x.ini --
[uwsgi]
xxtest=foo
xxbaz=quux
socket=:3030
wsgi-file=test-opt.py

# test-opt.py --
import uwsgi, pprint
pprint.pprint(uwsgi.opt)
application = lambda a, e: None

# Output of “bin/uwsgi x.ini” --
{'socket': ':3030',
'wsgi-file': 'test-opt.py',
'xxbaz': 'quux',
'xxtest': 'foo'}

From: uwsgi-bounces< at >lists.unbit.it [mailto:uwsgi-bounces< at >lists.unbit.it] On Behalf Of C Anthony Risinger
Sent: Tuesday, May 21, 2013 6:59 AM
To: uWSGI developers and users list
Subject: Re: [uWSGI] Access to the config options at the global scope in uWSGI app

On Mon, May 20, 2013 at 1:26 PM, dracek mracek <dracekm< at >gmail.com<mailto:dracekm< at >gmail.com>> wrote:
Hello,

I would like to ask if it's possible to access to config values defined in application config (ie .ini). I want mysql db connection was specified in the config but didn't find how to access to them i

Re: HTTP router and http-keepalive

Roberto De Ioris — 2013-05-21T05:27:02


Keep-alive on request with body shoulw now work.

Remember: every parsing error (both on request and response) will close
the connection

Re: Access to the config options at the global scope inuWSGI app

C Anthony Risinger — 2013-05-21T03:59:22


you can use `--get` for external access, but within python you can access
from the `uwsgi` module... i forget the exact attr off the top of my head,
but try:


...instead of `len` and methinks it will all pan out from there ;)

Re: sharing objects amongst workers

est — 2013-05-21T02:45:09

per-worker.

Actually, technically you can.

For posix only cross-worker connection sharing, use sendmsg() to pass the
MongoDB connection fd from one worker to another.

Only if the uWSGI master supports sendmsg() between workers and mules

Just an idea.





On Sat, May 18, 2013 at 1:44 PM, Roberto De Ioris <roberto-5KDOxZqKugI< at >public.gmane.org> wrote:

_______________________________________________
uWSGI mailing list
uWSGI-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Re: Django - SMTP

Venkatraman S — 2013-05-21T01:40:40

Should this be easier with spooler instead of celery?

-V
< at >venkasub <http://about.me/venkasub>



On Mon, May 20, 2013 at 10:39 PM, Łukasz Mierzwa <l.mierzwa-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>wrote:

_______________________________________________
uWSGI mailing list
uWSGI-FfzAktRlpg7/cILp9QSAqw< at >public.gmane.org
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Re: uwsgi emperor with multiple python versions

Jon Chappell — 2013-05-20T23:29:33

Of course, no sooner did I send this than I finally figured out that the
python 2.7 plugin had been compiled wrong!  I fixed that and it's now
working.

Thanks!


On Mon, May 20, 2013 at 7:16 PM, Jon Chappell <jon-1S8IvtbpCFD1P9xLtpHBDw< at >public.gmane.org> wrote:

uwsgi emperor with multiple python versions

Jon Chappell — 2013-05-20T23:16:54

Hi,

We are trying to run uwsgi with multiple python versions and are currently
compiling from source, but have also tried installation via pip.  We're on
Ubuntu 10.04 which has Python 2.6 as the default Python version, but need
to run a mix of 2.6 and 2.7 applications that should be managed by the
emperor.  The 2.6 application works fine, but the 2.7 application gets an
error when the emperor tries to start the workers.  The Python 2.6 and 2.7
plugins were both compiled also.

The init script being used for uwsgi is:
description "uWSGI"
start on runlevel [2345]
stop on runlevel [06]

respawn

exec uwsgi --logto /var/log/uwsgi/emperor.log --emperor /etc/uwsgi/conf.d/

The ini file in question:

[uwsgi]

socket=/tmp/uwsgi.sock
chmod-socket=660
pythonpath=/var/www/deployment
module=wsgi:application
master=True
pidfile=/tmp/deployment-master.pid
uid=uwsgi
gid=fm
harakiri=30
max-requests=5000
post-buffering=4096
logto=/var/log/uwsgi/deployment.log
harakiri-verbose=false
buffer-size=32768
virtualenv=/var/virtuale

Problem with the listen queue size

Andrew Knapp — 2013-05-20T21:51:16

Hello,

During some recent digging around in our uwsgi logs, I noticed that we
often get a few of these messages when we rollout code and restart the
workers:

"*** uWSGI listen queue of socket 3 full !!! (101/100) ***"

Looking around, it became clear that we needed to up the listen queue size
in our config, and I did, setting it to 1024 (and also upping the somaxconn
sysctl variable to 5120 on our machines). Now, when the workers start up,
they say they have a queue of 1024:

"your server socket listen backlog is limited to 1024 connections"

Unfortunately, even though it prints that, it doesn't seem to be taking
that value correctly. We still get error messages exactly like the one
above:

"*** uWSGI listen queue of socket 3 full !!! (101/100) ***"

I'm guessing the second part of "101/100" should be 1024, but it's not
showing up. I should also note that we are running this app in a emperor
setup. Is there something else we need to tune here?

Version of uWSGI: 1.2
Config file (I've hidden some variables,

Re: Access to the config options at the global scope inuWSGI app

dracek mracek — 2013-05-20T19:26:20

Thank you Roberto and Łukasz,

I will try it.

Tom

On 20. 5. 2013, at 20:35, "Roberto De Ioris" <roberto< at >unbit.it> wrote:

_______________________________________________
uWSGI mailing list
uWSGI< at >lists.unbit.it
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Re: Access to the config options at the global scope in uWSGI app

Roberto De Ioris — 2013-05-20T18:35:22



uwsgi.opt is a dictionary exporting all of the parsed options

Re: Access to the config options at the global scope inuWSGI app

Łukasz Mierzwa — 2013-05-20T18:31:34

can't you use --env for that? seems to be The Most Obvious Way for this
kind of things


2013/5/20 dracek mracek <dracekm-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>

Access to the config options at the global scope in uWSGIapp

dracek mracek — 2013-05-20T18:26:42

Hello,

I would like to ask if it's possible to access to config values defined in
application config (ie .ini). I want mysql db connection was specified in
the config but didn't find how to access to them in Python (using uWSGI
1.9.10, Debian)

So far I've gone through dozens of values but nothing seems to fit
configuration.
import uwsgi
len(uwsgi)

I also tried to build the uWSGI binary with using option embed_config in
buildconf/base.ini but building didn't succeed.

Before using ConfigParser in Python I'd like to check if there is an easier
solution in uWSGI :)

Thanks for any advice,

Re: Django - SMTP

Łukasz Mierzwa — 2013-05-20T17:09:18

It seems that it takes a lot of time to connect from django node to smtp
server and --harakiri kills your worker.
Verify with django "./manage.py shell" that you can send those emails with
your current code.
After that I would suggest to move smtp handling to background queue using
celery and https://pypi.python.org/pypi/django-celery-email


2013/5/20 Daniel Kopka <daniel.kopka-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>

Django - SMTP

Daniel Kopka — 2013-05-20T17:02:22

Hi,

When I try to send email from Django 1.2 via uWSGI in log I receives (uwsgi
1.9.10):

uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 248]
IOError: write error
*** HARAKIRI ON WORKER 6 (pid: 30863, try: 1) ***
HARAKIRI: -- syscall> 42 0x8 0x7fff3d23f710 0x10 0xe 0x1 0x100
0x7fff3d23f6b0 0x7fe481bcb8ad
HARAKIRI: -- wchan> inet_stream_connect
*** backtrace of 30863 ***
xx_uWSGI worker 6(uwsgi_backtrace+0x29) [0x451b49]
xx_uWSGI worker 6(what_i_am_doing+0x17) [0x451fc7]
/lib64/libc.so.6(+0x32920) [0x7fe480095920]
/lib64/libpthread.so.0(__connect+0x2b) [0x7fe481bcb8ab]
/srv/venvs/xx/lib64/python2.6/lib-dynload/_socketmodule.so(+0x6018)
[0x7fe47de2c018]
/srv/venvs/xx/lib64/python2.6/lib-dynload/_socketmodule.so(+0x809c)
[0x7fe47de2e09c]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x4e26) [0x7fe4807025c6]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927) [0x7fe480704657]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304) [0x7fe480702aa4]
/usr/lib64/libpython2.6.so.1.0(PyEval