Best c questions in October 2011

What does the C ??!??! operator do?!

192 votes

I saw a line of C that looked like this:

!ErrorHasOccured() ??!??! HandleError();

It compiled correctly and seems to run ok. It seems to like it's checking if an error has occurred, and if it has, it handles it, but I'm not really sure what it's actually doing or how it's doing it. It does look like the programmer is trying express his feelings about errors.

I have never seen the ??!??! before in any programming language, and I can't find documentation for it anywhere. (Google doesn't help with search terms like ??!??!). What does it do and how does the code sample work?

It's a trigraph that translates to |. So it says:

!ErrorHasOccured() || HandleError();

which, due to short circuiting, is equivalent to:

if (ErrorHasOccured())
    HandleError();

Guru of the Week (deals with C++ but relevant here), where I picked this up.

Possible origin of trigraphs or as @DwB points out in the comments it's more likely due to EBCDIC being difficult (again). This discussion on the IBM developerworks board seems to support that theory.

From ISO/IEC 9899:1999 §5.2.1.1, footnote 12 (h/t @Random832):

The trigraph sequences enable the input of characters that are not defined in the Invariant Code Set as described in ISO/IEC 646, which is a subset of the seven-bit US ASCII code set.

What's the point of const pointers?

74 votes

I'm not talking about pointers to const values, but const pointers themselves.

I'm learning C and C++ beyond the very basic stuff and just until today I realized that pointers are passed by value to functions, which makes sense. This means that inside a function I can make the copied pointer point to some other value without affecting the original pointer from the caller.

So what's the point of having a function header that says:

void foo(int* const ptr);

Inside such a function you cannot make ptr point to something else because it's const and you don't want it to be modified, but a function like this:

void foo(int* ptr);

Does the work just as well! because the pointer is copied anyways and the pointer in the caller is not affected even if you modify the copy. So what's the advantage of const?

Update: There are illegal sites like programmersgoodies.com linking to this question / answer. Sites like these are violating the StackOverflow attribution requirements.


Answer:

const is a tool which you should use in pursuit of a very important C++ concept:

Find bugs at compile-time, rather than run-time, by getting the compiler to enforce what you mean.

Even though it doesn't change the functionality, adding const generates a compiler error when you're doing things you didn't mean to do. Imagine the following typo:

void foo(int* ptr)
{
    ptr = 0;// oops, I meant *ptr = 0
}

If you use int* const, this would generate a compiler error because you're changing the value to ptr. Adding restrictions via syntax is a good thing in general. Just don't take it too far -- the example you gave is a case where most people don't bother using const.

Efficiency of premature return in a function

61 votes

This is a situation I encounter frequently as an inexperienced programmer and am wondering about particularly for an ambitious, speed-intensive project of mine I'm trying to optimize. For the major C-like languages (C, objC, C++, Java, C#, etc) and their usual compilers, will these two functions run just as efficiently? Is there any difference in the compiled code?

void foo1(bool flag)
{
    if (flag)
    {
        //Do stuff
        return;
    }

    //Do different stuff
}

void foo2(bool flag)
{
    if (flag)
    {
        //Do stuff
    }
    else
    {
        //Do different stuff
    }
}

Basically, is there ever a direct efficiency bonus/penalty when breaking or returning early? How is the stackframe involved? Are there optimized special cases? Are there any factors (like inlining or the size of "Do stuff") that could affect this significantly?

I'm always a proponent of improved legibility over minor optimizations (I see foo1 a lot with parameter validation), but this comes up so frequently that I'd like to set aside all worry once and for all.

And I'm aware of the pitfalls of premature optimization... ugh, those are some painful memories.

EDIT: I accepted an answer, but EJP's answer explains pretty succinctly why the use of a return is practically negligible (in assembly, the return creates a 'branch' to the end of the function, which is extremely fast. The branch alters the PC register and may also affect the cache and pipeline, which is pretty minuscule.) For this case in particular, it literally makes no difference because both the if/else and the return create the same branch to the end of the function.

There is no difference at all:

=====> cat test_return.cpp
extern void something();
extern void something2();

void test(bool b)
{
    if(b)
    {
        something();
    }
    else
        something2();
}
=====> cat test_return2.cpp
extern void something();
extern void something2();

void test(bool b)
{
    if(b)
    {
        something();
        return;
    }
    something2();
}
=====> rm -f test_return.s test_return2.s
=====> g++ -S test_return.cpp 
=====> g++ -S test_return2.cpp 
=====> diff test_return.s test_return2.s
=====> rm -f test_return.s test_return2.s
=====> clang++ -S test_return.cpp 
=====> clang++ -S test_return2.cpp 
=====> diff test_return.s test_return2.s
=====> 

Meaning no difference in generated code whatsoever even without optimization in two compilers

What does y -= m < 3 mean?

43 votes

While looking through some example C code, I came across this:

y -= m < 3;

What does this do? It it some kind of condensed for loop or something? It's impossible to google for as far as I know.

m < 3 is either 1 or 0, depending on the truth value.

So y=y-1 when m<3 is true, or y=y-0 when m>=3

What is normalized UTF-8 all about?

39 votes

The ICU project (which also now has a PHP library) contains the classes needed to help normalize UTF-8 strings to make it easier to compare values when searching.

However, I'm trying to figure out what this means for applications. For example, in which cases do I want "Canonical Equivalence" instead of "Compatibility equivalence", or vis-versa?

Everything You Never Wanted to Know about Unicode Normalization

Canonical Normalization

Unicode includes multiple ways to encode some characters, most notably accented characters. Canonical normalization changes the code points into a canonical encoding form. The resulting code points should appear identical to the original ones barring any bugs in the fonts or rendering engine.

When To Use

Because the results appear identical, it is always safe to apply canonical normalization to a string before storing or displaying it, as long as you can tolerate the result not being bit for bit identical to the input.

Canonical normalization comes in 2 forms: NFD and NFC. The two are equivalent in the sense that one can convert between these two forms without loss. Comparing two strings under NFC will always give the same result as comparing them under NFD.

NFD

NFD has the characters fully expanded out. This is the faster normalization form to calculate, but the results in more code points (i.e. uses more space).

If you just want to compare two strings that are not already normalized, this is the preferred normalization form unless you know you need compatability normalization.

NFC

NFC recombines code points when possible after running the NFD algorithm. This takes a little longer, but results in shorter strings.

Compatibility Normalization

Unicode also includes many characters that really do not belong, but were used in legacy character sets. Unicode added these to allow text in those character sets to be processed as Unicode, and then be converted back without loss.

Compatibility normalization converts these to the corresponding sequence of "real" characters, and also performs canonical normalization. The results of compatibility normalization may not appear identical to the originals.

Characters that include formatting information are replaced with ones that do not. For example the character gets converted to 9. Others don't involve formatting differences. For example the roman numeral character is converted to the regular letters IX.

Obviously, once this transformation has been performed, it is no longer possible to losslessly convert back to the original character set.

When to use

The Unicode Consortium suggests thinking of compatibility normalization like a ToUpperCase transform. It is something that may be useful in some circumstances, but you should not just apply it willy-nilly.

An excellent use case would be a search engine since you would probably want a search for 9 to match .

One thing you should probably not do is display the result of applying compatibility normalization to the user.

NFKC/NFKD

Compatibility normalization form comes in two forms NFKD and NFKC. They have the same relationship as between NFD and C.

Any string in NFKC is inherently also in NFC, and the same for the NFKD and NFD. Thus NFKD(x)=NFD(NFKC(x)), and NFKC(x)=NFC(NFKD(x)), etc.

Conclusion

If in doubt, go with canonical normalization. Choose NFC or NFD based on the space/speed tradeoff applicable, or based on what is required by something you are interoperating with.

Bizarre use of conditional operator in Linux

31 votes

In the 3.0.4 Linux kernel, mm/filemap.c has this line of code:

retval = retval ?: desc.error;

I've tried compiling a similar minimal test case with gcc -Wall and don't get any warnings; the behavior seems identical to:

retval = retval ? retval : desc.error;

Looking at the C99 standard, I can't figure out what formally describes this behavior. Why is this OK?

As several others have said, this is a GCC extension, not part of any standard. You'll get a warning for it if you use the -pedantic switch.

The point of this extension is not really visible in this case, but imagine if instead it was

retval = foo() ?: desc.error;

With the extension, foo() is called only once. Without it, you have to introduce a temporary variable to avoid calling foo() twice.

Why does GCC pad functions with NOPs?

26 votes

I've been working with C for a short while and very recently started to get into ASM. When I compile a program:

int main(void)
  {
  int a = 0;
  a += 1;
  return 0;
  }

The objdump disassembly has the code, but nops after the ret:

...
08048394 <main>:
 8048394:       55                      push   %ebp
 8048395:       89 e5                   mov    %esp,%ebp
 8048397:       83 ec 10                sub    $0x10,%esp
 804839a:       c7 45 fc 00 00 00 00    movl   $0x0,-0x4(%ebp)
 80483a1:       83 45 fc 01             addl   $0x1,-0x4(%ebp)
 80483a5:       b8 00 00 00 00          mov    $0x0,%eax
 80483aa:       c9                      leave  
 80483ab:       c3                      ret    
 80483ac:       90                      nop
 80483ad:       90                      nop
 80483ae:       90                      nop
 80483af:       90                      nop
...

From what I learned nops do nothing, and since after ret wouldn't even be executed.

My question is: why bother? Couldn't ELF(linux-x86) work with a .text section(+main) of any size?

I'd appreciate any help, just trying to learn.

First of all, gcc doesn't always do this. The padding is controlled by -falign-functions, which is automatically turned on by -O2 and -O3:

-falign-functions
-falign-functions=n

Align the start of functions to the next power-of-two greater than n, skipping up to n bytes. For instance, -falign-functions=32 aligns functions to the next 32-byte boundary, but -falign-functions=24 would align to the next 32-byte boundary only if this can be done by skipping 23 bytes or less.

-fno-align-functions and -falign-functions=1 are equivalent and mean that functions will not be aligned.

Some assemblers only support this flag when n is a power of two; in that case, it is rounded up.

If n is not specified or is zero, use a machine-dependent default.

Enabled at levels -O2, -O3.

There could be multiple reasons for doing this, but the main one on x86 is probably this:

Most processors fetch instructions in aligned 16-byte or 32-byte blocks. It can be advantageous to align critical loop entries and subroutine entries by 16 in order to minimize the number of 16-byte boundaries in the code. Alternatively, make sure that there is no 16-byte boundary in the first few instructions after a critical loop entry or subroutine entry.

(Quoted from "Optimizing subroutines in assembly language" by Agner Fog.)

edit: Here is an example that demonstrates the padding:

// align.c
int f(void) { return 0; }
int g(void) { return 0; }

When compiled using gcc 4.4.5 with default settings, I get:

align.o:     file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <f>:
   0:   55                      push   %rbp
   1:   48 89 e5                mov    %rsp,%rbp
   4:   b8 00 00 00 00          mov    $0x0,%eax
   9:   c9                      leaveq 
   a:   c3                      retq   

000000000000000b <g>:
   b:   55                      push   %rbp
   c:   48 89 e5                mov    %rsp,%rbp
   f:   b8 00 00 00 00          mov    $0x0,%eax
  14:   c9                      leaveq 
  15:   c3                      retq   

Specifying -falign-functions gives:

align.o:     file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <f>:
   0:   55                      push   %rbp
   1:   48 89 e5                mov    %rsp,%rbp
   4:   b8 00 00 00 00          mov    $0x0,%eax
   9:   c9                      leaveq 
   a:   c3                      retq   
   b:   eb 03                   jmp    10 <g>
   d:   90                      nop
   e:   90                      nop
   f:   90                      nop

0000000000000010 <g>:
  10:   55                      push   %rbp
  11:   48 89 e5                mov    %rsp,%rbp
  14:   b8 00 00 00 00          mov    $0x0,%eax
  19:   c9                      leaveq 
  1a:   c3                      retq   

Why was mixing declarations and code forbidden up until C99?

22 votes

I have recently become a teaching assistant for a university course which primarily teaches C. The course arbitrarily standardized on C90, probably due to widespread compiler support. One of the very confusing concepts to C newbies with previous Java experience is the rule that variable declarations and code may not be intermingled within a block (compound statement).

This limitation was finally lifted with C99, but I wonder: does anybody know why it was there in the first place? Does it simplify variable scope analysis? Does it allow the programmer to specify at which points of program execution the stack should grow for new variables?

I assume the language designers wouldn't have added such a limitation if it had absolutely no purpose at all.

In the very beginning of C the available memory and CPU resources were really scarce. So it had to compile really fast with minimal memory requirements.

Therefore the C language has been designed to require only a very simple compiler which compiles fast. This in turn lead to "single pass compiler" concept: The compiler reads the source-file and translates everything into assembler code as soon as possible - usually while reading the source file. For example: When the compiler reads the definition of a global variable the appropriate code is emitted immediately.

This trait is visible in C up until today:

  • C requires "forward declarations" of all and everything. A multi pass compiler could look forward and deduce the declarations of variables of functions in the same file by itself.
  • This in turn makes the *.h files necessary.
  • When compiling a function, the layout of the stack frame must be computed as soon as possible - otherwise the compiler had to do several passes over the function body.

Nowadays no serious C compiler is still "single pass", because many important optimizations cannot be done within one pass. A little bit more can be found in Wikipedia.

The standard body lingered for quite some time to relax that "single pass" point in regard to the function body. I assume, that other things were more important.

Clean and type-safe state machine implementation in a statically typed language?

20 votes

I implemented a simple state machine in Python:

import time

def a():
    print "a()"
    return b

def b():
    print "b()"
    return c

def c():
    print "c()"
    return a


if __name__ == "__main__":
    state = a
    while True:
        state = state()
        time.sleep(1)

I wanted to port it to C, because it wasn't fast enough. But C doesn't let me make a function that returns a function of the same type. I tried making the function of this type: typedef *fn(fn)(), but it doesn't work, so I had to use a structure instead. Now the code is very ugly!

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

typedef struct fn {
    struct fn (*f)(void);
} fn_t;

fn_t a(void);
fn_t b(void);
fn_t c(void);

fn_t a(void)
{
    fn_t f = {b};

    (void)printf("a()\n");

    return f;
}

fn_t b(void)
{
    fn_t f = {c};

    (void)printf("b()\n");

    return f;
}

fn_t c(void)
{
    fn_t f = {a};

    (void)printf("c()\n");

    return f;
}

int main(void)
{
    fn_t state = {a};

    for(;; (void)sleep(1)) state = state.f();

    return EXIT_SUCCESS;
}

So I figured it's a problem with C's broken type system. So I used a language with a real type system (Haskell), but the same problem happens. I can't just do something like:

type Fn = IO Fn
a :: Fn
a = print "a()" >> return b
b :: Fn
b = print "b()" >> return c
c :: Fn
c = print "c()" >> return a

I get the error, Cycle in type synonym declarations.

So I have to make some wrapper the same way I did for the C code like this:

import Control.Monad
import System.Posix

data Fn = Fn (IO Fn)

a :: IO Fn
a = print "a()" >> return (Fn b)

b :: IO Fn
b = print "b()" >> return (Fn c)

c :: IO Fn
c = print "c()" >> return (Fn a)

run = foldM (\(Fn f) () -> sleep 1 >> f) (Fn a) (repeat ())

Why is it so hard to make a state machine in a statically typed language? I have to make unnecessary overhead in statically typed languages as well. Dynamically typed languages don't have this problem. Is there an easier way to do it in a statically typed language?

If you use newtype instead of data, you don't incur any overhead. Also, you can wrap each state's function at the point of definition, so the expressions that use them don't have to:

import Control.Monad

newtype State = State { runState :: IO State }

a :: State
a = State $ print "a()" >> return b

b :: State
b = State $ print "b()" >> return c

c :: State
c = State $ print "c()" >> return a

runMachine :: State -> IO ()
runMachine s = runMachine =<< runState s

main = runMachine a

Edit: it struck me that runMachine has a more general form; a monadic version of iterate:

iterateM :: Monad m => (a -> m a) -> a -> m [a]
iterateM f a = do { b <- f a
                  ; as <- iterateM f b
                  ; return (a:as)
                  }

main = iterateM runState a

Edit: Hmm, iterateM causes a space-leak. Maybe iterateM_ would be better.

iterateM_ :: Monad m => (a -> m a) -> a -> m ()
iterateM_ f a = f a >>= iterateM_ f

main = iterateM_ runState a

Edit: If you want to thread some state through the state machine, you can use the same definition for State, but change the state functions to:

a :: Int -> State
a i = State $ do{ print $ "a(" ++ show i ++ ")"
                ; return $ b (i+1)
                }

b :: Int -> State
b i = State $ do{ print $ "b(" ++ show i ++ ")"
                ; return $ c (i+1)
                }

c :: Int -> State
c i = State $ do{ print $ "c(" ++ show i ++ ")"
                ; return $ a (i+1)
                }

main = iterateM_ runState $ a 1

How does this C code using function pointers work?

20 votes

I've found this code here : Printing 1 to 1000 without loop or conditionals

But i didn't understand how it works help me please.

#include <stdio.h>
#include <stdlib.h>

void main(int j) {
  printf("%d\n", j);
  (&main + (&exit - &main)*(j/1000))(j+1);
}

For j<1000, j/1000 is zero (integer division). So:

(&main + (&exit - &main)*(j/1000))(j+1);

is equivalent to:

(&main + (&exit - &main)*0)(j+1);

Which is:

(&main)(j+1);

Which calls main with j+1.

If j == 1000, then the same lines comes out as:

(&main + (&exit - &main)*1)(j+1);

Which boils down to

(&exit)(j+1);

Which is exit(j+1) and leaves the program.

Don't ever write code like that.


(&exit)(j+1) and exit(j+1) are essentially the same thing - quoting C99 §6.3.2.1/4:

A function designator is an expression that has function type. Except when it is the operand of the sizeof operator or the unary & operator, a function designator with type "function returning type" is converted to an expression that has type "pointer to function returning type".

exit is a function designator. Even without the unary & address-of operator, it is treated as a pointer to function. (The & just makes it explicit.)

And function calls are described in §6.5.2.2/1 and following:

The expression that denotes the called function78) shall have type pointer to function returning void or returning an object type other than an array type.

So exit(j+1) works because if the automatic conversion of the function type to a pointer-to-function type, and (&exit)(j+1) works as well with an explicit conversion to a pointer-to-function type.

That being said, the above code is not conforming (main takes either two arguments or none at all), and &exit - &main is, I believe, undefined according to §6.4.6/9:

When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; ...

Pointer initialization in C

Asked on Sun, 02 Oct 2011 by Bruce c
17 votes

In C why is it legal to do

char * str = "Hello";

but illegal to do

int * arr = {0,1,2,3};

I guess that's just how initializers work in C. However, you can do:

int *v = (int[]){1, 2, 3}; /* C99. */

Why is the minimalist, example Haskell quicksort not a "true" quicksort?

17 votes

Haskell's website introduces a very attractive 5-line quicksort function, as seen below.

quicksort [] = []
quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater)
    where
        lesser = filter (< p) xs
        greater = filter (>= p) xs

They also include a "True quicksort in C".

// To sort array a[] of size n: qsort(a,0,n-1)

void qsort(int a[], int lo, int hi) 
{
  int h, l, p, t;

  if (lo < hi) {
    l = lo;
    h = hi;
    p = a[hi];

    do {
      while ((l < h) && (a[l] <= p)) 
          l = l+1;
      while ((h > l) && (a[h] >= p))
          h = h-1;
      if (l < h) {
          t = a[l];
          a[l] = a[h];
          a[h] = t;
      }
    } while (l < h);

    a[hi] = a[l];
    a[l] = p;

    qsort( a, lo, l-1 );
    qsort( a, l+1, hi );
  }
}

A link below the C version directs to a page that states 'The quicksort quoted in Introduction isn't the "real" quicksort and doesn't scale for longer lists like the c code does.'

Why is the above Haskell function not a true quicksort? How does it fail to scale for longer lists?

EDIT: Here is the link at which this quicksort definition can be found. http://www.haskell.org/haskellwiki/Introduction#Quicksort_in_Haskell

The true quicksort has 2 beautiful aspects:

  1. Divide and conquer; break the problem into two smaller problems.
  2. Partition the elements in-place.

The short Haskell example demonstrates (1), but not (2). How (2) is done may not be obvious if you don't already know the technique!

How Are C Arrays Represented In Memory?

15 votes

I believe I understand how normal variables and pointers are represented in memory if you are using C.

For example, it's easy to understand that a pointer Ptr will have an address, and its value will be a different address, which is the space in memory it's pointing to. The following code:

int main(){
    int x = 10;
    int *Ptr;
    Ptr = &x;
return 0;
}

Would have the following representation in memory:

+---------------------+-------------+---------+
| Variable Name       | Address     | Value   | 
+---------------------+-------------+---------+
| x                   | 3342        | 10      |
+---------------------+-------------+---------+
| Ptr                 | 5466        | 3342    |
+---------------------+-------------+---------+

However I find it difficult to understand how arrays are represented in memory. For example the code:

int main(){
    int x[5];
        x[0]=12;
        x[1]=13;
        x[2]=14;

    printf("%p\n",(void*)x);
    printf("%p\n",(void*)&x);

return 0;
}

outputs the same address twice (for the sake of simplicity 10568). Meaning that x==&x. Yet *x (or x[0] in array notation) is equal to 12, *(x+1) (or x[1] in array notation) is equal to 13 and so on. How can this be represented? One way could be this:

+---------------------+-------------+----------+----------------------+
| Variable Name       | Address     | Value    | Value IF array       |
+---------------------+-------------+----------+----------------------+
| x                   | 10568       | 10568    | 12                   |
+---------------------+-------------+----------+----------------------+
|                     | 10572       |          | 13                   | 
+---------------------+-------------+----------+----------------------+
|                     | 10576       |          | 14                   | 
+---------------------+-------------+----------+----------------------+
|                     | 10580       |          | trash                | 
+---------------------+-------------+----------+----------------------+
|                     | 10584       |          | trash                | 
+---------------------+-------------+----------+----------------------+

Is this close to what happens, or completely off?

An array is a block of contiguous objects with no spaces in between. This means that x in your second example is represented in memory as:

+---------------------+-------------+---------+
| Variable Name       | Address     | Value   | 
+---------------------+-------------+---------+
| x                   | 10568       | 12      |
|                     |             +---------+
|                     |             | 13      |
|                     |             +---------+
|                     |             | 14      |
|                     |             +---------+
|                     |             | ??      |
|                     |             +---------+
|                     |             | ??      |
+---------------------+-------------+---------+

That is, x is five ints big, and has a single address.

The weird part about arrays isn't in how they're stored - it's how they're evaluated in expressions. If you use an array name somewhere that it isn't the subject of the unary & or sizeof operators, it evaluates to the address of its first member.

That is, if you just write x, you will get a value 10568 with type int *.

If, on the other hand you write &x, then the special rule doesn't apply - so the & operator works like it normally does, which means that it fetches the address of the array. In the example, this will be a value 10568 with type int (*)[5].

The reason that x == &x is that the address of the first member of an array is necessarily equal to the address of the array itself, since an array starts with its first member.

Is there a preferred way to order floating-point operands?

14 votes

Suppose I have a very small float a (for instance a=0.5) that enters the following expression:

6000.f * a * a;

Does the order of the operands make any difference? Is it better to write

6000.f * (a*a);

Or even

float result = a*a;
result *= 6000.f;

I've checked the classic What Every Computer Scientist Should Know About Floating-Point Arithmetic but couldn't find anything.

Is there an optimal way to order operands in a floating point operation?

It really depends on the values and your goals. For instance if a is very small, a*a might be zero, whereas 6000.0*a*a (which means (6000.0*a)*a) could still be nonzero. For avoiding overflow and underflow, the general rule is to apply the associative law to first perform multiplications where the operands' logs have opposite sign, which means squaring first is generally a worst strategy. On the other hand, for performance reasons, squaring first might be a very good strategy if you can reuse the value of the square. You may encounter yet another issue, which could matter more for correctness than overflow/underflow issues if your numbers will never be very close to zero or infinity: certain multiplications may be guaranteed to have exact answers, while others involve rounding. In general you'll get the most accurate results by minimizing the number of rounding steps that happen.

Filling the data in an Android bitmap as quickly as possible from C

13 votes

I've managed to get an android.graphics.Bitmap created and I'm successfully filling it via the SetPixels command.

The problem is that I start off with RGBA data. I then create a jintArray. I then call SetIntArray (effectively memcpying the data into the buffer). Then, finally, I call setPixels to actually set the pixels (which presumably causes another copy).

one big issue with doing this is that whether I used R8G8B8A8 or R5G6B5 or A8 I still have to convert my pixel data to R8G8B8A8 data.

Ideally I'd like a way to fill the buffer using only one copy and allow me to do it without doing the pixel format conversion.

Is there any way to directly get at the buffer data contained in a Bitmap? I see there is a function GetDirectBufferAddress in the JNI, but the documentation I can find on it suggests its limited to a java.nio.buffer. Can I directly get at the pixel data using this function? Perhaps by getting the internal buffer used by the Bitmap class?

Is my only way of using this to create a Global Ref'd Java.nio.buffer then each time I want to update, copy my pixel data into it and then use copyPixelsFromBuffer? This still involves 2 copies but can, at least, eliminate the pixel format change. Is this going to be any more efficient than the method I'm already using?

Is there an even better way of doing it?

Btw, I AM aware of the fact I can use the functions in < android/bitmap.h > but I would really like to not lose support for Android 2.1 and Android 2.2 ...

Cheers in advance!

Here comes a dirty yet working solution, working from Android 1.5 up to 4.0. Code is in C++.

                              //decls of some partial classes from Skia library
class SkRefCnt{
public:
   virtual ~SkRefCnt(){}
private:
   mutable int fRefCnt;
};

//----------------------------

class SkPixelRef: public SkRefCnt{
public:
   virtual class Factory getFactory() const;
   virtual void flatten(class SkFlattenableWriteBuffer&) const;
protected:
   virtual void* onLockPixels(class SkColorTable**) = 0;
   virtual void onUnlockPixels() = 0;
public:
   void *GetPixels(){
      SkColorTable *ct;
      return onLockPixels(&ct);
   }
};

jobject java_bitmap;  //your Bitmap object
jclass java_bitmap_class = env.GetObjectClass(java_bitmap);
class SkBitmap;
SkBitmap *sk_bitmap = (SkBitmap*)env.CallIntMethod(java_bitmap, env.GetMethodID(java_bitmap_class, "ni", "()I"));
SkPixelRef *sk_pix_ref;
sk_pix_ref = (SkPixelRef*)((int*)sk_bitmap)[1];
// get pointer to Bitmap's pixel memory, and lenght of single line in bytes
int buffer_pitch = env.CallIntMethod(java_bitmap, env.GetMethodID(java_bitmap_class, "getRowBytes", "()I"));
void *buffer = sk_pix_ref->GetPixels();

Using C macros to create C-style code which maps to Objective-C message calls?

6 votes

I'm sure I'll get 20 people saying "why would you want to do that anyways"... but, I'm going to ask my question none-the-less because it's somewhat academic in nature.

I'd like to use C macros to redefine [ClassName new] into something like: new(ClassName), and I'm wondering how to do this. I'm not super comfortable with C macros to begin with (I know - embarrassing - I should be) - and I'm definitely not comfortable mixing them in with my Objective-C code. So, on with the question...

First, being it's a preprocessor thing, can I do a simple substitution like such:

#define new(x) [x new]

or, for whatever reason, do I need to drop down to the objective-c runtime, and do something more akin to:

#define new(x) objc_msgSend(class_createInstance(x, 0), sel_registerName("init"))

What are the downfalls of doing something like this?

Is this kind of thing used often by others, or, would someone look at it and say "what the heck are you doing there"? (and should I care)

Thanks

EDIT:

It occurred to me after posting this, that I have, in fact, see this kind of thing before - in the Three20 lib, where they do things like this:

#define TT_RELEASE_SAFELY(__POINTER) { [__POINTER release]; __POINTER = nil; }
#define TT_INVALIDATE_TIMER(__TIMER) { [__TIMER invalidate]; __TIMER = nil; }

// Release a CoreFoundation object safely.
#define TT_RELEASE_CF_SAFELY(__REF) { if (nil != (__REF)) { CFRelease(__REF); __REF = nil; } }

So probably my question becomes simply; What are the downfalls of doing this, and is it a relatively accepted practice, or something that's going to get me into more trouble than it's worth?

Macros are processed first, and they are operating on the plain text source code. So yes, you can have your new macro generate Objective-C syntax or plain C syntax or even invalid syntax if you like.

The downfalls of using a macro in general is that, because it is parsed and processed in a separate step, it's possible to write macros that don't behave how you expect even when everything looks fine.

For example, this macro:

#define MAX(x,y) x > y ? x : y

Looks OK, but say you used it like this:

z = MAX(a,MAX(b,c));

It would be expanded by the preprocessor into something like this:

z = a > b > c ? b : c ? a : b > c ? b : c;

Which won't actually give you the max of the three arguments. To solve this you need to liberally sprinkle parenthesis in your macro definition, even where you don't think it is needed:

#define MAX(x,y) ((x) > (y) ? (x) : (y));

That fixes it, except I added a semicolon to the end, which is an understandable habit from writing a lot of C code, except now our macro expands to:

z = ((a) > (((b) > (c) ? (b) : (c));) ? (a) : (((b) > (c) ? (b) : (c));));;

Syntax errors!

If you look at how MAX is actually defined in Objective-C, it's pretty mess, but that's what you have to do to write macros safely. And you also need to consider that:

z = MAX(expensiveComputation(), reallyExpensiveComputation())

Will, unlike a function, actually execute one of those functions twice, unless you use a trick in your macro to basically emulate parameter passing.

So, to answer your question, yes it is totally possible, but writing safe macros is really hard. And you're doing this so you can pretend your Objective-C code is actually code in another language... Why would you want to do that, anyways?

Execute Python Script as Root (seteuid vs c-wrapper)

5 votes

I have a quick one off task in a python script that I'd like to call from Django (www user), that's going to need to root privileges.

At first I thought I would could use Python's os.seteuid() and set the setuid bit on the script, but then I realized that I would have to set the setuid bit on Python itself, which I assume is big no no. From what I can tell, this would also be the case if using sudo, which I really would like to avoid.

At this point, I'm considering just writing a C wrapper the uses seteuid and calls my python script as root, passing the necessary arguments to it.

Is this the correct thing to do or should I be looking at something else?

sudo does not require setuid bit on Python. You can enable sudo for one command only, no arguments:

 www          ALL=(ALL)       NOPASSWD:  /root/bin/reload-stuff.py ""

This would be secure if your script does not take any arguments, cannot be overridden by www user, and sudo does "env_reset" (the default in most distros).

You can accept arguments, but be very careful with them -- do not take output filenames, make sure you verify all inputs. In this case, remove "" from the end of sudo line.

How to store a database inside an Arduino?

5 votes

I'm working on project where I use an Arduino with a Bluetooth module and my cellphone Samsung Galaxy S II with Android OS. The idea of the project is to send some commands from my cellphone to the Arduino via Bluetooth. I want to include a database into the Arduino so that when I send login information from my cellphone, the Arduino will check the database and if the login information matches, it retrieves some data from the database and sends it to my cellphone.

How can I store a database inside the Arduino? Should I purchase an external EEPROM or RAM? And how can I deal with that database (adding, deleting and manipulating data)?

My Ardunio is of type UNO, BTW.

Just for simple login you don't need a database, you probably need just a simple table.

Consider first of all that usually EEPROMs allow from 1000 to 100000 write cycles. It means, if you write a single cell more than 100000 you have an high probability that your cell die, you cannot write it anymore.

The question is, how many logins are allowed? It is a all matter of choosing the right data structure and understand what is the amount of required memory.

Knowing the computational power of Arduino: If logins are just 2 .. 50, a simple list would be sufficient. Insertion at the end is O(1), deletion is O(n), lookup is O(n). A linked list however will allow you to reduce the number of writes for deletion to a constant small value.

If logins are more, 50 .. 1000, a sorted array with binary search is enough. Insertion is O(n), deletion is O(n), lookup is O(n log n). However the number of writes is O(n) both for deletion and insertion, and since writing is slow and can burn cells, it depends on the number of updates you want to do.

If logins are 1000 or more a binary tree is good. Insertion is O(n log n), deletion is O(n log n), lookup is O(n log n). The good thing is that for insertion and deletion you just need a small, constant number of writes.

Also an hashtable is good, but they usually use more memory. Insertion is averaged O(1), deletion is averaged O(1), lookup is averaged O(1). Insertion and deletion requires only a small constant number of write operations, less than a binary tree. As I said, this data structure uses more memory, speed comes at a cost.

You don't need a real relational database, but probably if you need too much users, you need an external EEPROM.

Of course, you have to save this data in a flash memory, internal or external, or you'll lose the database when you reset or power down the machine.

We can also say that you don't need to store the username and password, you can just store an hash of the password and username. If the hashed username and password exists, then you can allow login. In this way you can use fixed size memory and less memory. You can use MD5, it is the Android phone that have to send the MD5 hash, that is, 16 bytes, so the Arduino must only check if that MD5 hash exists in the list of users there, for example. And this is easy and fast.